Co-Worker Apparently Can't Stand Us

The workplace at my day job consists of a large room.  It's split in half, even though there is no barrier separating us.  One of them does one grade, the other another.  Within each half are six or seven rows.  Each row has three tables set up end-to-end.  Each table has enough space for a laptop and chair.

My row has only five people ... and as of today, there are only four.  The guy sitting next to me decided he wanted to move to the back table.  I would be crushed by the rejection, but the guy seems to be kind of a creep.

We helped each other score some tests.  I, naturally, am fast.  I didn't know he was deliberate/slow.  At one point this man, who usually wore an untucked polo and shorts and has close cropped hair and a poker face, stopped me and said that he wasn't reading as fast as I was.  I apologized, but the way he said, the tone, grabbed my attention.  He didn't say that with any undercutting humor.  He seemed a bit peeved.

The woman on the other side of this guy talked to him a little more than I did, so she went back there and asked him what's up.  She told me he just wanted to move back there, where there is only one other person working, and she too left her original workstation (which was the front row, the row ahead of me, in fact the laptop right in front of me).  I actually worked with her a little bit, and she said she moved back there because it was too noisy where she was.  That excuse I can understand.  What he told her?  No.  I'm pretty sure he was just sick of me and the guy next to me constantly getting up to pee or get food or just walk around.

He's a standoffish character.  He's got the same mien of a guy who kills and dismembers prostitutes, or at the very least goes home and beats his girlfriend.  It's the quiet ones that you have to watch out for.