In general, it's not recommended to touch your pet fish with bare hands. Fish have a slime coat, that protects them from bacteria, parasites, and helps them swim. Touching can potentially remove this layer, making fish more susceptible to infections.
However this is a big debate and it is up to you. I personally do not do it and I also wash my hands very well without soap before putting my hands in a tank.