May 302012
 

SQL Saturday #119, our third one in Chicago, was a total blast. This was only my second time on the organizing committee, but I’ve yet to hear anything negative about the event. From my perspective, everything went incredibly smoothly!

SQL Saturdays are a learning experience for all involved – not just the attendees. Rather than give my usual play-by-play of the weekend, I thought I would share the things I learned and how I’ll make it even more amazing next year:

  • I am not a good predictor of coffee consumption. Full disclosure: I’m also not a coffee drinker. I ordered 25 boxes of coffee (1 box = 10 cups), 15 of regular and 10 decaf. We ended up with about 8 left over, of which 7 were decaf. I’ll try to order a few less next year so there’s less waste.
  • Those in attendance were much more interested in roast beef sandwiches than mostaccioli or salad.  That’s not to say they didn’t get eaten, but we ended up with a ton left over. It’s obviously better to have too much food as opposed to too little, but I’d probably order slightly less of them next time if we ended up going with Portillo’s again.
  • Portillo’s Catering is amazing. They were a pleasure to work with and I wouldn’t hesitate to order from them again. Their crew was at the event on time as promised and had everything set up and ready to go very quickly.
  • Running a food truck is a rough business. Last year’s caterer, Mr. Meatyballs, was every bit as timely and efficient as Portillo’s above, however he no longer operates a truck. Running a food truck is hard work, but it’s especially difficult in Chicago, where city ordinances are incredibly unfriendly towards them. Chef Foss wrote an excellent blog post describing what it was like running the Meatyballs Mobile, and why he ultimately scrapped the idea for a sit-down restaurant instead. I applaud his openness and wish him every success in this endeavor.
  • Small sessions can be a good thing. My talk on data compression was during the first timeslot of the day, which I was sharing with some heavy hitters. I ended up with only about 10 people in my room when it came time to start at 9:15, which had me feeling pretty down. About halfway through I realized that this was actually a very good thing – everyone present was genuinely interested in data compression and many were planning on implementing it in the near future. I got some excellent questions and was able to answer all of them, as well as start a few discussions that went a bit deeper than my slide deck covered. It always feels great to fill a room, but it’s even better to know you were able to help everyone that was present.
  • Student Labs were a great success! We tried something new this year and worked with DeVry University to offer some lab classes to their students. This is the second time DeVry has donated the use of their facility, and we were happy to be able to give something back. I wasn’t able to attend these sessions, but the blog posts of those who did are very positive!
  • Mark Vaillancourt (blog | @MarkVSQL) does a great Gilbert Gottfried impression. as witnessed at the after-party.
  • Unicorn masks can get pretty freaky!

Unicorn Mask

Thanks again to everyone for their hard work:

  • The organizing committee for putting it all together: Ted Krueger (blog | @onpnt), Bill Lescher (@blescher), Aaron Lowe (blog | @vendoran), Wendy Pastrick (blog | @wendy_dance), Rich Rousseau (blog | @zigzag219), and myself.
  • The speakers, who gave their time and knowledge to make our schedule awesome.
  • The attendees, without whom there would be no reason to have an event.
  • And last but certainly not least, our sponsors. Without their financial support, none of this could have happened.

I hope to see everyone again next year!

May 242012
 

A week ago I was privileged to attend Brent Ozar’s Free-Con event. The original Free-Con was held in Seattle before the 2010 PASS Summit, and the second one took place in Chicago before last year’s SQL Saturday. Since it was so awesome last year, Brent decided to do it again.

What is the Free-Con?
To paraphrase Brent’s explanation, one of the awesome things about events like SQL Saturday and the PASS Summit is that you have so many great community contributors assembled in one place. Unfortunately since there’s an event going on, there isn’t always a chance for everyone to get together and talk. The Free-Con was created to address this issue and facilitate discussion about a wide variety of topics, none of which include SQL Server.

So what did we talk about? Brent had a slide deck prepared, but it wasn’t a traditional presentation – more like ideas and a few short videos to spark discussion amongst ourselves. Some of the topics for discussion included:

– It’s believed to take approximately 10,000 hours of experience to master any subject.
Skills Timeline Resumes: a very interesting way to express your work history and see how your skillsets have evolved over time
– Lean Methodology: minimizing waste and getting the minimum viable product to market as quickly as possible
– Supergraphics: A picture says a thousand words, but these say way more.
– Where do we find inspiration?
– How can we improve SQL Saturday tomorrow?

Getting ready for Free-Con!

Enjoying some Pre-Free-Con breakfast

Location
The Free-Con was held at the Catalyst Ranch, a space that’s every bit as unique as the free-con itself! It’s the perfect location to get the creative juices flowing, with all kinds of awesome colors, toys, games, and props. They also have excellent food – their breakfast buffet is tremendous. We also had deep dish pizza from Giordano’s for lunch!

Highlights
For me, the highlight of the day was being able to share a room with so many amazing, talented, passionate people and be able to bounce ideas off of them. Since SQL Server really wasn’t on the agenda, it was an excellent chance to talk about things we do outside of work, family, hobbies, and the like. When you have a group of people talking about things that really inspire them, it makes for a whole lot of awesome, and that’s what I love so much about the free-con.

May 152012
 

On April 2 of this year, the National Archives released the complete population schedule of the 1940 census. These records were highly anticipated not only for their genealogical value but also because of their detailed information about an incredibly interesting period of U.S. History. This census captured the point in time where the country was finally starting to come out of the great depression but had not yet entered World War II. Many questions it included were new and designed to gauge the effects of the depression, with topics including income, education, unemployment, and migration. In 1940, millions were employed by the WPA, PWA, and other New Deal agencies, and the Farm Security Administration’s photography program had a small group of photographers traversing the nation capturing images of everyday American life. Some of my favorite photos come from this collection.

Population Schedule Form

Population Schedule Form (Click to Enlarge)

As interesting as the Census is for all its historical and social reasons, there’s an equally awesome tale to be told of all its data and the technologies behind it. Setting up a table in SQL Server to store 310 million rows and aggregate results from them would be pretty easy today – many DBAs deal with tables that are orders of magnitude larger than that, but in 1940 it was a major undertaking involving thousands of workers. Today the census is still a non-trivial task, however I’d imagine most of that work goes into getting data from the population into a database, while calculating the results from that is relatively simple.

ETL: Enumerate, Tabulate, Lock Up
Prior to 1960, censuses weren’t mailed to your house like they are now. Instead every household was visited in person by an enumerator, a single person responsible for an “Enumeration District”, or “ED”. EDs varied wildly in size, and could consist of a single block in a large city, or an entire township in a more rural area. The enumerator would stop by and ask questions about each member of your household while writing the answers onto a population schedule form that measured 23.75″ wide by 18.5″ high. Yep, the entire database was on paper. Torn page detection must have been a very serious issue! When the enumerator had information about every last person in their district, they would send their data to the Census Office via log shipping Registered Mail.

Once in Washington, an army of operators transferred each record from the population schedule forms to a punch card. Punch cards had been used to tabulate the census since 1890 and were still the best technology available fifty years later. The 1880 census was tabulated by hand, which took 7 years to complete. Knowing there had to be a better way to calculate results, former census employee Herman Hollerith set out to create a machine that could count results from data stored on punched cards. He won a contract to tabulate the 1890 census, which was completed in only 1 year. By 1900, he had formed the Tabulating Machine Company and greatly increased his fees, knowing he had a monopoly and the Census Office would have no other option than to pay them. By 1910, census employees had developed and patented their own counting machine to avoid using Hollerith’s. The Tabulating Machine Company, which by then had merged and changed names to the Computer Tabulating Recording Company, was nearly bankrupted by the loss of business. They eventually got their act together and were able to turn a profit. In 1924, Computer Tabulating Recording Company changed its name to International Business Machines Corporation.

Tabulating by Machine

Women in (1940) Technology: Tabulating By Machine

After the records were copied to punch cards and tabulated by machine, the aggregated results were released immediately for uses like determining congressional seats and allocation of public funds. Since the population schedules contain information on individuals, they are held for 72 years before being released for research purposes. Rather than keep all 3.9 million pages of records on paper, the Census Office used the most compressed format available at the time, microfilm. Apparently they had not yet discovered the rather obscure and undocumented BACKUP CENSUS TO TAPE=’MICROFILM’ WITH COMPRESSION; command. The records released this year are images scanned from that microfilm.

Indexing
Since all the data consists of images, there’s no easy way to index them. Optical character recognition software is pretty good these days, but probably not good enough to pick out the handwriting in these images – most of which is in cursive. Instead everything was indexed by enumeration district, meaning you need to know where someone was living during April of 1940 before you can search for them. Many genealogy websites are now working on indexing this data by name, but it is not expected to be completed for a few months.

Finding Your Family
If you had relatives in the US in 1940 and know where they lived at that time, I highly recommend looking for them. Everything can be found for free at http://1940census.archives.gov. The first thing you’ll need to do is find which enumeration district they lived in. If you have an address, you are very much in luck. If you only have a general idea of where they were, then you’ll probably have to do a bit more work to find them. The census site lets you drill down by state, county, and city, and provides a list of EDs that apply. If you’re searching in an urban area you might need to use maps and/or descriptions returned by the search to narrow down exactly which ED they were in. If the official site isn’t finding anything for you, I’ve also had luck using Steve Morse’s 1940 ED Finder. Once armed with a list of relevant enumeration districts, you can view or download the population schedules from each district and look for people you recognize. You’ll probably end up looking through all the sheets because the entries on the forms aren’t always in order. My guess is that enumerators would start going down a street, skip houses where nobody was at home and then come back to them later.

I was fortunate enough to find all of my family, and it’s really neat to be able to see a snapshot of their lives at a time when my grandparents were close to my age. It also gave me great appreciation for what a chore recordkeeping was in that era! Even if you have no relatives in this census I think it’s still worth taking a look at – it was very interesting to see what kinds of jobs people had, their education level, and how much they were paid. My family was in the suburbs of Chicago at that time, and probably 7 out of 10 people in their area worked in “telephone manufacturing” which would have been at Western Electric’s Hawthorne Works. My wife’s family was in a rural area downstate, and practically everyone worked on a farm and the few who didn’t were employed by the WPA. The best job title I saw when searching for her family was “chicken picker”.

Best of luck if you end up searching for your ancestors. If you find any who were employed as a chicken picker, let me know!

Fun Videos

May 032012
 

Chicago’s third annual SQL Saturday will soon be upon us, and as food and beverage coordinator it’s my job to obsess over what’s on the menu.

SQL Saturday food offerings seem to have evolved over time. The first few I attended offered pizza, which is of course a great way to feed a large number of people. While very cost-effective, pizza is also not all that exciting. This did not go unnoticed; it seems there was a shift to more local food specialties that aren’t pizza. Lately I’ve seen more local offerings like amazing barbecue in Kansas City and excellent bratwurst in Wisconsin.

BeefThis year I’m happy to announce that lunch for SQL Saturday Chicago will be Portillo’s! We’ll have Italian Beef sandwiches, Mostaccioli, and Grilled Veggie Sandwiches for those who have requested a vegetarian option. Be sure to bring your appetite!

Also if you haven’t already seen, our schedule has been posted! We have some excellent speakers lined up. Now is a great time to start thinking about what sessions you’d like to attend.

Pre-Cons
If you’re looking to extend your weekend a bit, we have two excellent pre-cons lined up for Friday, May 18:

These pre-cons are priced at $99 each and can be booked by emailing sqlsaturday119@sqlsaturday.com.

We’re looking forward to another wonderful event! Can’t wait to see you there in two weeks!