JM2 User Test Report

Description:

JM2 Ag and Cattle is a company devoted to providing products and services from and for the beef cattle industry. The team works to provide wholesome and safe agricultural products and services at affordable prices by being good stewards of God's resources. The company has been operating since 2009. JM2 has grown from simply producing beef and providing artificial insemination services to producing hay, woodcraft, and many new services.

The website is designed to give users information abut beef and the cattle industry, provide a means to purchase their products, and potentially order services. The site includes links to recipes as well as links to informative sites about beef and the beef cattle industry.

Objective:

The primary objective for this e-commerce site is to be capable of leading a user through the entire purchasing experience with ease, speed and a feeling of satisfaction

Questions:

Will the site lead the user through the process in a reasonable amount of time without confusion?
Will the site encourage the user to buy products that they may not have previously considered?
Will the site keep users from leaving without buying?

Methods:

The definition of evolution is the process of systematically collecting data that inform us about what it is like for a particular user or group of users to use a product for a particular task in a certain type of environment (2). This means that we will need to test this prototype in a manner that allows the user to express their experience in the most open way possible. The only way we can get accurate data is to allow free expression from the users after a complete experience.

This is a formative evaluation of our prototype before completion of the fully designed site. We will be testing using the Axure cloud file of our prototype on a laptop. Users will be selected to represent different demographic groups. To test this objective we will be using a couple of different methods. We will use observations of the testers running through the site. The testers will be recorded for later review during the evaluation of feedback. They are asked to comment as they go through the site, but also asked guided questions later in a survey format.

Practical Issues:

Ideally, this process would take a few days, and would use multiple testers from each demographic group. There would also be enough of a budget to offer incentives to testers. Our current issues are a lack of budget, time constraints, and a lack of tester availability. Therefore, testers will be volunteers, and people that we know. There will also only be one male and female from each age group.

Ethical Issues:

As testers will be recorded, there will be a consent form provided for users to sign with the assurance that no personal information will be stored or revealed, only demographic. This means that age, gender, and preferences only will be used. Also, the audio recording will not be used beyond evaluation reference, and will not have any video. Quotes only will be taken from the recordings.

Evaluation:

Once the consent form is signed, the user will be take to a room set up like a home office to complete the test on a laptop. They wll be given a persona or scenario and a task to complete.

This persona will be similar to their own, but intended to help guide the user through the scenario with a purpose. After the persona, the user is given a task to test. They are assured that they are not being tested, but useing this persona and task t evaluate the objective for us. We ask the user to read all of this out loud for the recording to ensure that the user is fully aware of the objective. They then run through the task without further instruction. If the user is capable, they provide comments about what they see, think, and feel through the task.

Finally, after they feel they have completed the task, the user is asked a series of questions intended to help guide their evaluation into information we can use and process to adjust our website. Some of these question will be simple to help guide the user into thinking about a particular topic. Others will be more complex and open ended to gain specific information on preferences and possible problems in the site.

User 1:

Persona:

You are a 53 year old male. You are comfortable with ordering online, but often prefer to buy the product in the store as you can be certain of what you just paid for. A family member referred you to a friend's website to buy beef asking for support for this local family business.

Task:

You decide to give it a try and get online to order meat for the next time your family gets together, and visit jm2agandcattle.com to order a few steaks.

Please verbally express your thoughts and actions as you work through the task. State everything you like and dislike upon reviewing the site, every action you take to accomplish your task, and your perception of the site as a whole. Remember you are not being tested, but are evaluating the quality of the site.

Notes/observations:

This user volunteered, but was very short in his test. He came across as though he didn't care to be doing the test. He did go through and look at different aspects of the site, but his responses were very short and even sarcastic. He stated that he didn't have much of an emotional response to the site. He added that he was able to complete the task, so he didn't think about much else.

Survey:

Design

Does the site seem aesthetically pleasing?
a. Yes
Upon viewing the site, how does it make you feel? On a scale of 1 to 5: 1 makes you want to leave the site, and 5 entices you to explore further.
a. 4
What specific design aspects did you like or dislike?
a. Pictures, the steaks look yummy!
How could we have changed the design to be more pleasing?
a. Scratch and sniff.

Process

Were you able to accomplish the task you were asked to test?
a. Yes
Did you feel the task was possible within a reasonable amount of time?
a. Yes
How long do you feel it took to complete the task? (1: an unacceptable amount of time, 5: faster than expected)
a. 4
How much did the site encourage you to complete the buying process? (1: you wanted to leave the site before completing the first step, 5: you would have considered buying more?)
a. 5
What did you like or dislike about the ordering process?
a. I like the minimal number of page changes
How could we improve this process?
a. Taste

Functionality

Was the site capable of performing the functions you needed to complete the task?
a. Yes
How easy did you feel the task was to complete? (1: it was next to impossible, 5: it was easier than expected)
a. 4
Did you find any portion of the process to be confusing, or were you unable to find information you needed to complete your task?
a. No
What did you like or dislike about the functionality available to you?
a. No dislikes.
How could we improve this process?
a. No improvements

User 2:

Persona:

You are a 50 year old female. You are very comfortable with ordering online, and make it a goal every year to get all of your Christmas gifts by ordering them online. A friend from work referred you to their brother’s website to buy beef asking for support for this local family business.

Task:

You decide to give it a try and get online to order meat for your family’s Father’s Day celebration as a treat for your husband, and visit jm2agandcattle.com to order a few steaks.

Notes/Observations:

User seemed very enthusiastic about being able to give input on the website. She took an extensive amount of time going through the pages and looking at the different portions of the design. She stated that the white background seemed harsh, and would have liked a calmer background like a pale grey, or pale tan. She commented her dislike for the layout of information and grammar on the homepage like the list of products. She would have preferred they be in a bulleted format. She stated that the reading didn’t impact her ability to accomplish the task, but it would likely have turned her off of the website if she had simply happened across it. Finally, she felt that the category page needed to have more space under each product, and the half orders needed to be listed under the quarter orders.

Survey:

Design

Does the site seem aesthetically pleasing?
Loved the picture! Immediately upon opening the site, I noticed the lovely pond scene and was intrigued. While looking at the geese, I noticed steak sale and was excited to catch it at the right time.
Upon viewing the site, how does it make you feel? On a scale of 1 to 5: 1 makes you want to leave the site, and 5 entices you to explore further.
Overall - 4
a. It looks very easy to navigate – 5
b. Reading, Yuk! – 1
I can skip the reading, so it didn’t count as much as the navigation.
What specific design aspects did you like or dislike?
a. Color
i. Loved the green. That is very much a down-home farm color.
ii. White is too harsh. I would have softened it with a beige or other light earthy color.
b. Description, Products, Services, Mission
i. Couple of typos
ii. Would have liked the Mission to be first
iii. Bold/Underline the titles
iv. Use bullets instead of sentences
How could we have changed the design to be more pleasing?
a. Lists instead of sentences
b. Larger font for main titles of meat choices
c. Larger separation between types of meat
d. Light color other than white for background. Too harsh.
e. Move the Half Options Between the Quarter Options and Whole Option.

Process

Were you able to accomplish the task you were asked to test?
Yes
Did you feel the task was possible within a reasonable amount of time?
Yes
How long do you feel it took to complete the task? (1: an unacceptable amount of time, 5: faster than expected)
4
How much did the site encourage you to complete the buying process? (1: you wanted to leave the site before completing the first step, 5: you would have considered buying more?) 4
a. The list of product categories at the top interested me in the wooden bowls as well.
b. Knowing that the business mission is to be a good steward of resources was right up my alley of going green and leaving behind a small carbon footprint.
What did you like or dislike about the ordering process?
a. I would have liked to see a “Continue Shopping” option rather than going directly to a payment page.
b. I wanted to go back to the home page but didn’t see a button for that. It took me a minute or two to think about clicking on the cow in the corner.
How could we improve this process?
a. The overall process was great.

Functionality

Was the site capable of performing the functions you needed to complete the task?
Not sure how this is different from ‘Process’, but I was able to do everything I needed to do.
How easy did you feel the task was to complete? (1: it was next to impossible, 5: it was easier than expected.
5
a. I don’t think of farms as very high-tech, so the efficiency of the website was a bit of a surprise.
Did you find any portion of the process to be confusing, or were you unable to find information you needed to complete your task?
Just the ‘Continue Shopping’ and ‘Home Page’ buttons.
What did you like or dislike about the functionality available to you?
See 13.
How could we improve this process?
I am not a very good cook. A link to some cooking times and processes might be very helpful.
Overall, very well done!

User 3:

Persona:

You are a 24 year old female. You are very comfortable with ordering online, and often use it to buy decorations for your new house. Your stumble onto jm2agandcattle.com because of the woodcraft, and see that they also sell beef.

Task:

Your husband’s birthday is coming up, and this is local meat. You decide to give it a try and order a few steaks.

Notes/Observations:

User was enthusiastic at the opportunity to help a local company produce a quality site. He did not commentate much initially, but did verbalize his thoughts once he started the survey. He stated that he had no emotional response initially. He was focused on ordering steaks, but when he read the first question started thinking about the visual design of the site. He also said that he wanted the clickable items to be more obvious. For example, the images lower on the homepage could have bold title text or hover states to help visually clarify the links.

Survey:

Design

Does the site seem aesthetically pleasing?
a. Yes, I appreciate the picture with the cuts of meat (cut sections of beef from each cow) as I don’t know which cuts of meat come from where.
b. Simple pictures, good logo
c. I would make the title of the cite bigger to emphasize the company (just make the font size bigger)
Upon viewing the site, how does it make you feel? On a scale of 1 to 5: 1 makes you want to leave the site, and 5 entices you to explore further.
a. 4, it appears to have good quality services and products from the home page, so it makes me want to look a little deeper into the meat. The meat page allows you to see all the cuts of meat and emphasizes that they are all properly packaged and flash-frozen for safety.
What specific design aspects did you like or dislike?
a. I would prefer to have a list (maybe bullets or numbers) on the products and services to more clearly see what is available from just a quick glance. I know you have bullets below to give a deeper description about the beef products, hay, etc. but it appears a little wordy initially under the bird picture
b. Liked the pictures (previously mentioned above); easy to navigate from the home page
How could we have changed the design to be more pleasing?
a. Spelling error (the hay) in the products section and misspelled “services” in Mission statement; misspelled “management” in services section on the homepage
b. Capitalize all first words of bullets under Beef Products
c. Increase font size or color of the company name to make it stand out

Process

Were you able to accomplish the task you were asked to test?
a. Yes, I was able to browse through the meat and find a product that was suitable for my husband’s birthday
Did you feel the task was possible within a reasonable amount of time?
a. Yes, it was easy to navigate the site and find what I was looking for; easy contact for questions and also easy process to order and input personal information
How long do you feel it took to complete the task? (1: an unacceptable amount of time, 5: faster than expected)
a. 4: very acceptable amount of time; I spent a little bit of time to browse the site as well
How much did the site encourage you to complete the buying process? (1: you wanted to leave the site before completing the first step, 5: you would have considered buying more?)
a. 5: would have considered purchasing more (especially the wood products) for home decoration!
What did you like or dislike about the ordering process?
a. I liked that it was all on one page; there was an option to have the shipping information be the same as billing (so I didn’t have to input my information twice); easy to adjust quantity of product being ordered
b. Would have liked to know when the Memorial Day Exclusive sale (10% off steak purchase) was ending and more details regarding the sale (are there limits on amount of steaks you can purchase, etc.)
How could we improve this process?
a. May show other purchasing options on the side? Customers who purchased this item also enjoyed …

Functionality

Was the site capable of performing the functions you needed to complete the task?
a. Yes
How easy did you feel the task was to complete? (1: it was next to impossible, 5: it was easier than expected)
a. 5, easy website to navigate, good tabs and fast/easy ordering section
Did you find any portion of the process to be confusing, or were you unable to find information you needed to complete your task?
a. No
What did you like or dislike about the functionality available to you?
a. Liked the ease of completion and how easy the steps were to follow to complete my order
How could we improve this process?
a. Nothing that I can think of at this time

User 4

Persona:

You are a 26 year old male. You are very comfortable with ordering online, and often use it to find new recipes. Being a bit of a foodie and taking pride in the quality of food you can make yourself, you go in search of a new beef supply…the grocery store meat just won’t do.

Task:

You find jm2agandcattle.com through a google search, and are intrigued at the idea of supporting a local producer as well. You decide to give it a try and order a few steaks.

Notes/Observations:

This user was also very enthusiastic about the chance to help a local producer. She was incredibly thorough, and had lots of comments on every question. She did not, however, comment while she was running through the site. She was very focused on her observations and had difficulty expressing her thoughts verbally while she completed her task.

Survey:

Design

Does the site seem aesthetically pleasing?
Yes
Upon viewing the site, how does it make you feel? On a scale of 1 to 5: 1 makes you want to leave the site, and 5 entices you to explore further.
4 – I would’ve liked to explore the wooden bowls and ag services links.
What specific design aspects did you like or dislike?
I wanted to click the title to go back to the home page.
How could we have changed the design to be more pleasing?
None

Process

Were you able to accomplish the task you were asked to test?
Yes
Did you feel the task was possible within a reasonable amount of time?
Yes, it took less than a minute to complete an order.
How long do you feel it took to complete the task? (1: an unacceptable amount of time, 5: faster than expected)
5
How much did the site encourage you to complete the buying process? (1: you wanted to leave the site before completing the first step, 5: you would have considered buying more?)
5
What did you like or dislike about the ordering process?
I liked the uniformity of the input process.
How could we improve this process?
None

Functionality

Was the site capable of performing the functions you needed to complete the task?
The process of ordering is exactly as it is on most online ordering sites. Easy to use and complete.
How easy did you feel the task was to complete? (1: it was next to impossible, 5: it was easier than expected)
5
Did you find any portion of the process to be confusing, or were you unable to find information you needed to complete your task?
I wouldn’t have known that the links on the bottom half of the page were clickable had I not hovered the mouse over them. So, either bold the title or underline, etc. to make it more obvious.
What did you like or dislike about the functionality available to you?
I like being able to see the info on the home page and easy to find.
How could we improve this process?
See number 13

User Summaries

User 1: This user is a 55 year old male. He is also the owner of the company being used as the subject of this project. He volunteered to run a test, but seemed irritated when he did the test. He was very short in his answers, and seemed almost as though he didn’t care about the outcome of the site. Had I the resources and the available time to gather more testers, his responses would possibly be taken out of the evaluation. His responses offer little help.

User 2: This user is a 53 year old female. She is the wife of the company owner, and is a life science and technology teacher. She was excited about the potential a new website might offer the company, and was eager to help. She made many suggestions, and noticed some problems with the website layout.

User 3: She is a 24 year old female, and personal friend. This user was excited about the opportunity to help our local company expand its reach. She found a few typos, and expressed several comments both positive and negative, as well as suggestions to make changes.

User 4: He is a 25 year old male, and also a personal friend. He has performed a user test for
me before, and was eager to do this one. This user gave a few suggestions as well, including some that are very well known theories in UX Design.

Survey Summaries:

Based on these ratings, I believe we still have some room for improvement. Keeping in mind that we cannot please everyone, it is still important to take a serious look at what is feasible and consider the suggestions from all user tests.

Design:

We could definitely make some improvements in this design. First, the reading on the homepage could stand a layout change. The list of products could be better organized, along with much of the other information on the home page. This would help increase the readability of the page. Tullis in 1987 (1) did a study looking at how the organization of information could affect a user’s ability to quickly find information for which they may be searching. They found that splitting the data into lines similar to bullets made it more readable, as opposed to a paragraph style of information. This could be applied to the homepage of this website by making the products and services a bulleted list. An image of this is shown below.

The next design change will be in the visibility of the links, or clickable items. Many sites use a hover state to demonstrate when items are clickable. Betsy Ross (3), another local producer, has a color change in her site, seen below:

She also has a dropdown, which I do not believe would be applicable on this site, but the black to blue color change in this hover state is obvious. Something similar like the dark green to a lighter green could be applied to the navigation.

It was also mentioned that the images lower on the home page be made more obvious as links. I think the hover state is still an option, but bolding the titles, and using an underline will help make these more obvious. Having the image enlarge slightly in a hover state would be one option. Another choice could be changing the title color. The second option would follow a theme, but the first option would add a little variety. I think doing both would be too much. The image associated with these changes is shown below.

Finally, it was suggested that the title of the page be larger. Another user thought it might be the link to the home page, so that is an option. Neither of these are necessarily common practice. The title is already a size 24 font. Since the navigation is also a size 24 font, it might be a good idea to make the title a bit larger. It would be possible to change the title into a link as well. It couldn’t hurt to have it available to those users that do not know the trend of the logo linking to the home page. In the image below, you can see that the title is the same size as the navigation links.

A few positives were noted on this page. First, the banner was appreciated by one user as a natural and welcoming feel to the page. Second, the images used helped entice the users to look further into the products. Finally, the information and images definitely encouraged the users to look for more in the site.

On the category page, it was pointed out that the item label is too small. Space is needed between the rows of items, and the larger orders need to be rearranged. Putting the half orders next to the quarters makes it harder to understand and even notice the half orders. It is better practice to place the half category below the quarter. This arrangement also follows the principle tested by Tullis (1). A positive comment about this page was that the images show the packaging. If you visit other beef sites, they often show the raw unpackaged products. One user noticed and commented on this.

Process:

A few positives were that the site encourages users to look further and into other products. Another is the minimal number of page changes. This user, after further questioning appreciated that all of the delivery and billing information was on one page. Two others mentioned the ease of having all of this information on one page, along with the uniformity within the page. A third is the mention of “being good stewards of God’s resources”. The user expressed that this aligned with her desires to leave a minimal footprint.

The suggested changes to the ordering process were for more information regarding the Memorial Day sale. Another user requested a “continue shopping” button be added on the shopping cart page. This is actually good practice as it encourages the user to buy more product. This is an important addition to our changes. This user also mentioned confusion about the logo being the home page link. This is the second comment about confusion in the logo home button. It might be reasonable to consider a button on the navigation bar to link to the home page. A few of the local pages, burgundypasturebeef.com for example, have a “Home” button on their navigation.

Functionality:

There was another mention of the “continue shopping” and “home” buttons. The “continue shopping” will be a definite addition. There is a request for more information about recipes and the beef industry. This is a page on the current site, and will be added to the site. This point in the project however, the focus was on e-commerce functionality. This page will likely be added to the final project. Lastly, there is another comment about the visibility of links on the home page. These will be made clearer with bolding and enlarging the titles, and possibly by adding a hover state to the image and description combination with enlargement.

The ease and intuition involved with the ordering steps was commented on more than once. The efficiency as well was brought up as something that impressed a user. Finally, every user commented on the ease of the navigation. Every link took the user exactly where they expected. The only confusion came from a lack of an obvious home button. Craik in 1943 (1) described a mental model as “internal constructions of some aspect of the external world enabling predictions to be made”. Creating an experience in which the users are able to predict what should happen is the primary goal. This way, they can complete the order process without confusion or frustration because they already have an idea of what should happen. From this feedback, it seems we can fully accomplish this goal with a few minor changes in the functionality.

In conclusion, there seems to be a very positive response to the design and usability. There are clearly still some changes that need to be made. The two primary changes for functionality are to include buttons allowing for more navigation ability, and make the links more obvious with hover states, bolds, and underlines. Finally, we need to make the site more readable. To do this, lists will be made into bullets instead of paragraph format. This will keep the readers from simply scanning and allow them to more easily see the items in the list.