Doug Hillmer April 29, 2007

Background:

Sometime in late March, Lori Weissberg, a reporter for the San Diego Union Tribune, contacted PIO and was referred to me with regard to a request she had for guidance in using ACS data. Her goal was to study characteristics of those people moving into San Diego County, CA compared with those leaving the county. They had already developed some hypotheses about certain characteristics, such as educational attainment.

Lori, and her data support person at the paper, David Washburn, were interested in learning if they could get the information they wanted from the ACS PUMS data. Because of the way the 2000 sample PUMA boundaries were drawn in California, there is asset of PUMAs that completely partitions San Diego county. (They assured me that there had been no changes in the county boundaries of any significance since 2000). They wanted to use the PUMS data directly to study the in- and out-migrants based on the 2005 ACS. They had two basic choices to start: download the PUMS data for the entire U.S. from AFF or use Ferrett to either download the data they needed for their study or create the tables they needed directly in Ferrett. Ferrett is definitely the more powerful tool for a taks such as this one. In order to study the out-migrants from a specific area, one really needs to search all the data for the entire country. Ferrett allows the user to filter the data that is downloaded in a manner tailored to the problem. AFF only allows the user to download the entire country or specific states; i.e., no other conditions may be imposed. Furthermore, Ferett does allow the user to do all their basic tabulations for such a study directly in the Ferrett application without downloading any data.

For these reasons, and because I wanted to see how well Ferrett worked as a tool for a more advanced task, such as in- and out-migration, I recommended that David Washburn learn how to use Ferrett and that I, and the Ferrett staff would assist him in getting the data and estimates he needed.

Assisting David Washburn in Using Ferrett:

It was not easy to help David Washburn get to the point where he could use Feret on hiw own. It required three separate phone calls, each of about an hour in length. In the first call, I conferenced in Bill Hazard of the Ferrett staff. Between phone calls, I gave Washburn some suggestions for things he might do and how to do them in Ferrett. He made an honest effort to do as much as he could on hiw own, but he definitely needed assistance to really see how Ferrett works. For example, it was not clear to him how to organize the estimates in a data table in Ferrett when he had three or more dimensions. The other demands on Washburn’s time forced him to stop working with Ferrett , making it even more difficult for him to reach a point where he could proceed on his own. This is going to be a constant reality with people in the media: they have very little time to devote to learning a tool such as Ferrett.

At the end of the third phone call, I had the impression that Washburn was understanding how Ferrett works and how to get what he needed. However, I’m still not sure he will succeed with Ferret. I plan to follow up with him soon to find out how far he actually got on this story.

Conclusion and Recommendations:

There are definitely some usability issues for people who are new to DataFerrett. This tool has a very different approach (the “Data Basket”) to creating an extract or a data tabulation. There are also many screens that could benefit from better wording or a change in the look and feel. In addition to the issues related to Ferrett itself, it was obvious that the ACS PUMS documentation was lacking in some areas, making it much more difficult for a novice user to understand how the PUMS data could be used to attack a problem such as this one. For example, the current documentation doesn’t really explain how to use the MIGPUMA field which is essential for studying in- an out-migration with the PUMS.

Therefore, as a follow-up activity to this de-briefing document, I recommend that an extended example document be created describing the variables on the PUMS that are needed and how to use them in Ferrett. This would very quickly point to the areas in the current PUMS documentation that need to be improved.

Attached below are some Ferrett screen shots from certain steps in this in- and out-migration project along with my comments on problems encountered by the user.

How does the user know to select “Selectable Geographies” versus “Geographic entities”?

User highlights PUMAs. Given the “dialogue” between the user and the app that will follow, it is not clear why the line directly above the PUMAs line regarding FIPS state code is needed – and, it is somewhat confusing to see two separate lines referring to FIPS state code.

How would the user know what to type in as the “name” of a PUMA? PUMAs are like tracts; they don’t really have names. If the user wants a specific PUMA, should the user type in the full 5-digit code with leading 0s or just as a number omitting leading 0s.?

Don’t say things like “Select ST from list” when you could just as easily write real English, like “Select state from list”. If you want to convey to the user that “ST” is the name of the field containing the state, use a construct like “state (ST)”.

From this point on, the computer-user dialogue is fairly clear.

Not clear that the contents of GEOG-101 limit the results of the tabulation even if I don’t display the PUMAs in GEG-101 in the table.

It is a very nice feature that Ferrett shows the total for a variable at the top. It would be even nicer if the label for a recode (in this case, RECODE2) could be used here.

-- CavanCapps - 30 Apr 2007

• HillmerUsability.doc: Doug Hillmer's comments working with reporters

• HillmerUsability.doc: Doug Hillmer's comments working with reporters