That's actually an easy number to calculate. The problem is the number is totally misleading. First let's answer the question straight up.
Those Not Working Full Time
The above very scary-looking chart shows the civilian non-instititional population minus those usually working full time.
Noninstitutional means those over the age of 16, not in prisons, mental institutions, etc.
The current non-institutional population is 129.61 million. At the beginning of 2009 it was 118.92 million and at the start of the recession in November of 2007 it was 111.06 million.
Since the beginning of the recession, the rise in the number of people not working fulltime is 18.55 million.
Calculation Discussion
The first problem with the above chart is that it fails to account for demographics. There is a huge rise in population as well as huge rise in boomer retirement.
Moreover, over time, there has been a legitimate rise in the number of people going to college after high school graduation.
Brief Word on the Unemployment Rate
As pertains to the much ballyhooed declining unemployment rate, there is also a non-insignificant number of people in college who really would rather be working.
Add to that, millions of people who would rather be working, but are not counted as unemployed because they stopped looking for a job.
That is why many, myself included, believe the unemployment rate is a joke.
Proper Comparison
To properly address Roger's question, one needs to factor in;
The problem in producing the proper calculation is insufficient data. The St. Louis Fed repository (named Fred), that I used to easily create the above chart, does not have the data.
Not even the BLS has the data we need to properly answer the question. Information on the critical age group 55 to 64 is scant or missing.
However, we can look at age group 16-54 or 25-54 for meaningful comparisons. I selected the latter because it filters out the rising trend of people going to college.
Using BLS data, I produced the following charts.
Not Working Full Time – Age Group 25-54