The Pinnawala Elephant Orphanage is a sanctuary for Indian elephants. It's a majestic sight to see, and you won't see as many and in greater numbers than this. These are elephants are treated as "wild", but they do have a routine. They're herded by their caretakers from the hills where they roam through the town to the river where they bath.
At the top of the hill is where the actual orphanage is. Babies are still bottle fed, and it attracts quite a crowd.
Watching from one of the plethora of restaurants at the river is probably the most interesting behavior-wise. You can see these beautiful creatures at play. Several will start wandering off across the river. They'll soon be herded back by the keepers, but it's interesting that they feel the urge to move.
I write this four years afterwards, and so I recall some of what I wrote that night. I had just flown in that day at 2am, slept from probably 4am to 8am, and was out the door shortly thereafter. Maybe this was to my advantage. It helped me cope with the jet lag. The sights were surreal as my head was in a slight daze all day. In any case, I'll quote from what I wrote about this place:
Elephants are awesome creatures. They work, play, eat, and sleep just like everyone else in Sri Lanka. I cannot say they are particularly cute (except babies), but they are awesome. Elephants eating leaves, walking to the river, cooling down, trying to get to the other side to eat, and "thinking".
Eating lunch and watching a heard of orphaned elephants playing in a river on a warm day with a nice breeze — the feeling of content that beauty surrounds you in many forms.
I finished after that by trying to compare this once-in-a-lifetime experience with that of seeing cherry blossoms fall. It wasn't an ephemeral beauty, but it was immensely calming and satisfying.
I'm sure there's a bus, but the best way of traveling to remote places is by car. Make sure you don't hire a tuk tuk. The train will be slow, but it's possible to catch a bus or taxi from a train station in Rambukkana.