Peter Levin’s Rethinking Markets

July 9, 2008

Types of variables, drop-down menus

Filed under: Data, Organizations — Peter @ 5:49 pm

Over at 37 Signals, they have a regular series detailing their design decisions. It is an insightful feature and an insightful blog.

Their latest discussion is about how they managed a question on their support forms. I want to drop some research methodology on this problem. While their discussion is about how to design a feedback form, it is also about the kinds of questions you should ask on a survey. And it would benefit from a discussion of categorical/nominal variables, ordinal variables, and interval-level variables. Yep.

So, terms. Categorical variables (also called nominal variables) are those variables with 2 or more ’states’, but without an intrinsic ordering. Male/female is categorical, as is eye or hair color, race, what school you went to. Ordinal variables are those variable with two or more states that have an ordering to them. Low/medium/high are ordinal variables. Less than HS, HS, some college, BA is ordinal. Interval-level variables are ordered, and the distance between categories is evenly spaced. Income, height, and years of education are all interval-level variables.

The difference between categorical/nominal variables and ordinal variables is the hierarchical ordering of the latter. What school you went to is nominal, but tiered ranking of what school you went to is ordinal. Tiered ranking may be ordinal, but amount of school endowment is interval-level. And quantitative variables are a hint that the variable is ordinal or interval, but not decisive (zip codes are nominal, for instance).

So, back to 37 Signals. What they want is some variable that would be easy to understand (from the customer perspective) and helpful to process (from the company perspective). Their first attempt looked something like this:

It takes time to think through what my state of mind is, because the items are almost ordered, but not really. Confused, worried, upset, and panicked are not points on a continuum, they are just different states of being. The question is asked as a categorical variable question. But it is one that they reallywanted to be an ordinal variable question. They tried to solve the problem by formalizing an ordinal variable, and putting a numbered ordering system on it to make it clearly so:

This is easier to deal with as a customer, since you can sort of pick up your relative state of panic. In other words, you pick up that there is a rough ordering very quickly, and the numbers help a lot in this respect. But alas, what’s good for the customer was no good for 37Signals. The reason is that while they began by wanting to know the subjectivity of the customer, what they really really wanted to know was, ‘how important is this problem for our company?’. Reasonable, but different. This is their final solution:

They have a reason for this, and it is a decent one:

Now, if something’s broken, we can spot it and fix it right away. A system failure is much more important to us (and our customers) than a feature request or general feedback. This method lets us prioritize these queries accordingly, instead of treating them like they’re all the same.

However, this final solution kind of sucks, I think. The problem is that it moves priority from the customer to the company, while giving an illusion of giving control to the customer. That is, the variable is categorical/nominal for the customer, but ordinal for the company. In other words, what is important to the company is more important than what is important to this particular customer. This is probably even more true for those customers for whom everything for them is the most important thing in the world. And yet.

I think perhaps a better solution splits the question into two, which provides space for both ‘urgency for the customer’ and ‘urgency for the organization’.

At the risk of adding yet another item to your survey/form, you have solved both problems with some cognitive ease: customers are defaulted to medium (which is easy for people to just ignore/skip over or shift with little cognitive difficulty), which would allow the organization to give its own priority to the categorical variable. If the customer changes the default to ‘not urgent’, this still stands. If the customer makes their own priority ‘urgent’, then the organization has some discretion on whether to treat this as ‘urgent for us’ or not, but at least has a sense of the panic level for the customer.

I’m sure there is an aesthetic here as well, but the general lesson should be two-fold. 1) Consider carefully the meanings behind your survey variables. Categorical variables often require thinking about, particularly as the category options become large. Ordinal and interval variables (which create an ordering) are easy, until they are too refined. Let’s say you are at a hospital, and a doctor asks you ‘how do you feel?’ Sorting through a list of adjectives that describe your feelings sucks. And assessing your level of pain between 1 and 7 is easier than between 1 and 1000.

The thing is, sometimes what is important to you is not what’s important to the doctor. If you feel throbbing, it’s not lethal. If you feel numbness, it is. In this case, the options are categorical to the patient, but ordinal to the doctor.

Which leads to 2) If you want your customers’ opinions, it may behoove you to give them a way to tell these to you. In the medical example, what kind of pain and how much does it hurt are two questions; patients care more about the second, even if doctors care more about the first. So don’t ask what kind of pain without asking how much does it hurt. Even if this saves the patient’s life, they will still be pissed at you for dismissing their subjective reality.

February 7, 2008

What is XBRL, and Who does XBRL help?

Filed under: Data, Institutional, Markets, Technology — Peter @ 10:02 am

Put it on your radar screens, the next big thing is going to be XBRL. It stands for extensible business reporting language, and it is meant to commensurate business reporting via standardization. So instead of entering text into an annual report, companies, governments, NGOs, anyone who would like to comply with governmental mandate will be using XBRL. You can think of XBRL as a set of metatags for financial and company data, so that instead of bracket-tags for header, title, links, etc. you would have bracket-tags for earnings, time periods, definitions of costs, etc.

From CoreFiling’s insight blog: “It won’t be very long before it is those documents - the bar-coded financial disclosures - that will be the primary materials consumed by financial market systems to help analysts and investors make decisions about the best way to invest. This is vastly more sophisticated than today’s processes that rely on slow and inaccurate re-keying of a subset of the financial information published by companies.”

This is commensuration more than just standardization, since the tags are designed to be specific to a particular business enough so that everyone is not required to give the same information, yet the tags are standardized enough that everyone is required to give information that can be made comparable. The pitch for companies (other than, because otherwise we’ll fine you and take away your business license) is that XBRL will make their financial reporting less costly, less prone to error, and ultimately more efficient.

Personally, I think this is a flat out misrepresentation of what’s going on here. XBRL helps one group of people orders of magnitude more than anyone else: investors. And the trade-off between increased government efficiency and business streamlining of compliance data on the one hand, and increased ability for data-gatherers for banks, hedge funds, and the investor class is totally totally off the charts. What this will end up doing is: 1) creating a standard way for companies to report financials; 2) creating some increased efficiency for government entities to keep tabs on the finances of these organizations; and 3) create a massive additional datastream for financial services and investment firms to work with. If you think it is a challenge for public firms to resist making short-term decisions based on financial analysts’ quarterly reports of earnings now, wait until this information is directly readable by quant trading models.

This would be an amazing dissertation topic. I would track: a) the creation of the standard; b) the adoption of the standard around the world; c) how XBRL is being incorporated into financial modeling; d) the before-and-after effects of XBRL on market prices for firms; and e) qualitatively, what gets excised from XBRL, or rather, what remains incommensurable about firms, governments, etc.

UBMatrix
XBRL’s main site
US SEC’s ‘Interactive Data Viewers’
Microsoft uses XBRL
US GAAP XBRL Taxonomy (GAAP is the accounting standard in the US)
CoreFiling