If you don't have SENTRI or Global Entry then, at the very least, get your Passport Card.  All three should shorten your crossing times.  

Otherwise I recommend you cross mid-afternoons between 2-4pm.....all lines are typically short.

Here's the opinion of some UCSD students that created an app for border wait times.

http://techfragments.com/free-iphone-app-best-time-to-cross-border/

You didn't say if you are crossing by car or on foot. Or at San Ysidro or Otay.  It matters as the best crossing times for each permutation is different.

For the record there are times it is quicker to cross at Otay by car and/or on foot.  Beyond that I'll limit my response to San Ysidro.

The worst times to cross (on foot and by car) are the morning commute hours when people who live in TJ and work in San Diego cross.  Sundays followed by Saturday are worse than weekdays to cross by foot or car.

Crossing by car on weekdays is often good around 5-6 pm. Followed by midnight to 2 am.

Crossing by foot on weekdays after about 7-8pm  is usually good.

The CBP provides telephone information by calling 619-690-8999.  You can also get information on-line at www.apps.cbp.gov/bwt.  All the apps I've ever found use information provided by CBP, which doesn't make them all that accurate.  My favorite app is Border Info.  (The icon has a green--not blue--background with driver wearing a seat belt.  The app with the blue background is Border Wait.)

The problem with CBP information is that (1) in actuality what it provides is an overly optimistic crossing estimate and (2) it is only updated hourly.  (Assuming they don't forget, which they often do.)

The best source of crossing information between 6am and 10pm comes from Telnor.  It is updated every 15 minutes and provides a physical count or cars and vehicles in line, instead of an estimated time.  The number is (664) 700-7000.  However, you'll need a Mexican cell phone to call Telnor and, of course, the information is provided in Spanish.