The Taoiseach said he hoped voters would not judge him on the basis of a criticised encounter with a disability care worker in Co Cork.