Monthly Archives: March 2011

Presentation by Brian Witlin on Applied Startup Metrics on 3.17.11 @ 500Startups

On March 22, 2011, The Lean Coffee Kelowna crew discussed Usability Testing, it’s goals and it’s types.

Goals of Usability Testing:

• Performance — How much time, and how many steps, are required for people to complete basic tasks? (For example, find something to buy, create a new account, and order the item.)

• Accuracy — How many mistakes did people make? (And were they fatal or recoverable with the right information?)

• Recall — How much does the person remember afterwards or after periods of non-use?

• Emotional response — How does the person feel about the tasks completed? Is the person confident, stressed? Would the user recommend this system to a friend?

Types of Usability Testing:

• Hallway Testing: This is a general methodology of usability testing. Rather than using an in-house, trained group of testers, just five to six random people, indicative of a cross-section of end users, are brought in to test the product, or service. The name of the technique refers to the fact that the testers should be random people who pass by in the hallway

• Remote Testing: In a scenario where usability evaluators, developers and prospective users are located in different countries and time zones, conducting a traditional lab usability evaluation creates challenges both from the cost and logistical perspectives. These concerns led to research on remote usability evaluation, with the user and the evaluators separated over space and time.Remote testing, which facilitates evaluations being done in the context of the user’s other tasks and technology can be either synchronous or asynchronous. Synchronous usability testing methodologies involve video conferencing or employ remote application sharing tools such as WebEx. The former involves real time one-on-one communication between the evaluator and the user, while the latter involves the evaluator and user working separately

• Expert Review: another general method of usability testing. As the name suggests, this method relies on bringing in experts with
  experience in the field (possibly from companies that specialize in usability testing) to evaluate the usability of a product.

• Think aloud: Users are asked to say whatever they are looking at, thinking, doing or feeling as they go about their task. This enables observers to see first-hand the process of task completion

Tools:

• crazyegg.com – starts at $9 a month
• fivesecondtest.com – Free
• silverbackapp.com – Free for 30 days, $65
• clixpy.com – $5 for 100 recorded sessions

Today (March 29, 2011), marked the fourth meeting of LeanCofffeeKL, and our topic of discussion was about MVP or Minimum Viable Product. These are notes for the meeting, so we can keep in mind what we discussed. A lot of these notes will be presented in bullet-point to keep it to the topic.

First, what is MVP?

“The minimum viable product is that version of a new product which allows a team to collect the maximum amount of validated learning about customers with the least effort.” – Eric Ries

One idea with MVP is learning the most you can by doing the least. Another suggestion regarding MVP is the least you can do to launch an idea and ship it, the better. MVP is aimed towards early adopters more than your end users. Early adopters are:

1. More forgiving

2. More likely to give feedback

3. Able to grasp product vision The difference between end users and early adopters are your early adopters may be a select group of
   users who specifically work in the industry you are targeting, generally you want users who can bring value to your testing. Using an analogy of building a car, do you plan the design of the car or it’s features first? Generally, you start with what the car absolutely needs: engine, wheels, controls, chasis, but the look of the chasis comes afterwards. Worry about what’s inside the container, before worrying about the container.

Techniques for an MVP:

• Product: (Smoke test) The canonical MVP strategy for a web application is to create a mock website for the product and purchase online advertising to direct traffic to the site. The mock website may consist of a marketing landing page with a link for more information or purchase. The link is not connected to a purchasing system, instead clicks are recorded and measure customer interest.

• Feature: (Deploy first, code later) A link to a new feature in a web application may be provided in a prominent location on an existing website. The feature is not implemented, rather an apology, mock-up, or marketing page is provided. Clicks of the link are recorded and provide an indication as to the demand for the feature in the customer base.

When planning out features for your MVP, you need to ask yourself the follow questions about each feature:

1. Does this [insert word here] achieve a pupose?

2. Is it worth it?

3. Do you struggle to determine how to execute a feature?

Resources, useful reading to learn more about MVP:

• http://en.wikipedia.org/wiki/Minimum_viable_product
• http://www.startuplessonslearned.com/2009/08/minimum-viable-product-guide.html
• http://www.startuplessonslearned.com/2009/03/minimum-viable-product.html
• http://radoff.com/blog/2010/05/04/minimum-viable-product-rant/

Presentation for the inaugural Lean Startup Meetup in San Francisco by Eric Ries on MVP