The bulk of this essential nutrient is made by the skin on exposure to sunlight. The rest comes from the diet.
A study in Clinical Endocrinology journal of 2,299 men found those with enough of the vitamin had more of the male sex hormone than those with less.
Blood levels of both dipped in the winter and peaked in the summer.
Low testosterone levels can impact on a man's libido as well as zap energy levels.
It also performs essential functions in both men and women such as maintaining muscle strength and bone density.
“ Men who ensure that their body is at least sufficiently supplied with vitamin D are doing good for their testosterone levels and their libido among other things ”
Ad Brand of the Sunlight Research Forum
The researchers from the Medical University of Graz, Austria, found men with at least 30 nanograms of vitamin D per millilitre of blood had much more testosterone circulating than those lacking in vitamin D.
Across the 2,299 men studied, testosterone and vitamin D levels appeared to peak in the month of August, and drop off in the winter, hitting their lowest levels in March.
Professor Winfried Marz and colleagues, who ran the study, said scientists should now look at whether vitamin D supplements would have the same effect on testosterone.
Ad Brand of the Sunlight Research Forum in The Netherlands, a non-profit organisation set up to inform the general public on latest medical research on sun and health, said: "Men who ensure that their body is at least sufficiently supplied with vitamin D are doing good for their testosterone levels and their libido among other things."
Cancer experts warn that too much sun exposure is damaging for the health.
Allan Pacey, senior lecturer in Andrology at the University of Sheffield, said: "We know that medically we can increase the libido and general well-being of men who have low levels of testosterone by giving them testosterone replacement therapy.
"However, this is for a defined set of medical circumstances where testosterone production is low.
"Whether healthy men notice a significant changes throughout the year is less clear and I would urge men to be sensible about using sunbeds in the winter months given the known risks of using them excessively."
Jessica Harris of Cancer Research UK said: "Enjoying the sun safely while taking care not to burn should help people strike a balance between making enough vitamin D and avoiding a higher risk of skin cancer.
"People can also top up their levels of vitamin D by eating more foods like oily fish such as salmon, trout or mackerel."