Solar panels are currently not highly efficient although efficiency continues to improve with technological advances. The efficiency can be dramatically affected by many factors, including regular, everyday issues like the amount of shade (from trees, structures...), wind, dust, snow, angle of the panels, amount of daylight, access to direct sunlight, solar intensity, and so on.
Storage is suggested, adding extra costs. You can use the solar energy generated from your panels directly (and even sell it to the grid, if your municipality has such a program in place) - but since the sun doesn't shine 24 hours a day, every day, then the solar energy needs to be "stored" in batteries for later use. Otherwise you would need to be connected to the grid as backup, or own a generator.
A relatively small amount of energy is generated by solar panels. Providing 100% of the energy required by the average home, for example, would require a vast array of solar panels. Not only is this impractical, it's very costly. Even if you were to build your own solar panel the costs would still be a challenge for many homeowners (a big reason why there's currently a lack of support for solar power).
Access to direct sunlight is needed for solar panels to performance at their best. Often, commercial installations of solar panels will readjust during the day to maintain an angle that provides them with direct sunlight - best for converting into energy. However, most homeowners simply mount solar panels in a rigid, fixed position, which means that the panels only have access to direct sunlight for a limited amount of time each day. Solar trackers are available to automatically re-orient the panels, however they can add significantly to the cost of the entire system.
Solar panels must be kept clean to operate at their most efficient. Keeping the panels clean and free of dust, dirt, and debris can be a big job, depending on where your panels are placed. And while panels often last 25 years or so, repairs can be required from time-to-time. Finally, what does one do with solar panels and batteries that are at the end of their useful life?
Currently, the limits of solar power appear to be many, and most eventually lead to the primary sticking point for homeowners: cost. While most of us would like to be able to do something beneficial for the environment, the investment costs can be daunting. However a good compromise would be to use solar power for small applications where it's cheap to install and maintain. In the future, the efficiency of solar cells will continue to improve and hopefully provide homeowners and businesses a viable alternative to drawing power from the grid.