SQL performance many-to-many with same table

SQL performance many-to-many with same table

I have the following tables (simplified) on SQL Server 2012 Standard edition:
CREATE TABLE Objects (ObjectID int, Class char(1))
and a connector table
CREATE TABLE Pointer (SourceID int, TargetID int)
which connects objects to one another.
I find that performance is really slow using this structure, and need tips
on what to do. There are about 10 different classes. A pointer can only
link different classes.