Writing Effective Assessments

Posted on September 29, 2015 by Seniye Groff

There are many types of assessments used to check learning. Generally, formative assessments are learning activities that appear throughout a course, whereas summative assessments usually appear at the end. The table below summarizes the differences.

Formative

(learning activities, challenges, learning aids, knowledge checks, guided practices, learning modules)

Summative

(activities, tests, quizzes, challenges, exams, assessments)

Low consequences for failure (learners can reverse decisions and responses)
Feedback prescriptive and instructive, guiding the learner
Used as a learning aid or practice
Can be used to diagnose areas where learners need to improve
Used as a basis for improvement or checkpoint before continuing to the next section
Feedback appears frequently, usually after a learner chooses a response or makes a decision
Usually appears at the end of a section and covers only one or two sections (non-cumulative)
Qualitative feedback

High consequences for failure (learners may have to retake the whole test)
Learners only given one chance to answer each question
Decisions (such as in a selling simulation) not reversible
Used to determine if learners pass or fail the course
Feedback is corrective (learner should know material by this point)
Feedback can appear after each question or cumulatively at the end
Usually appears at the end of a course or after many sections (cumulative)
Qualitative or quantitative feedback (depending on number and nature of previous assessments)

Match assessment types and questions with the outcome you want

The best place to start in your assessment is with your learning objectives. In fact, you can map out your assessment questions before even writing the course as a way to keep the questions focused on the objectives. What behavior or outcome do you want to affect? This is where it’s useful to pull out the Bloom’s Taxonomy and decide which level of learning you want for each assessment question or section.

Try to focus on the behavior you’re trying to affect. For example, say a learning objective calls for sales reps to know enough about product specs to make the right recommendations to customers. You may find yourself writing assessment questions that test learners’ knowledge of product specs, but is memorization the behavior you’re looking for? Probably not. What you probably want the sales rep to do is know how and where to find those specs. In that case, your assessment would instruct learners to use the sales tools at hand to find the right specs instead of requiring them to memorize the specs.

Tips

The following tips are guidelines for knowing what to do and what to avoid when writing summative assessments:

Map your questions to the course’s learning objectives.
Feel free to use prescriptive feedback (especially if the course is light on activities).
Keep feedback short.
Make sure correct answers are similar in length to incorrect responses (i.e., avoid the tendency to make the correct answer the longest one).
Make all possible responses reasonable (second-guess some incorrect responses that learners might come up with).
Use humor sparingly.
Avoid constructions like “Which of the following is NOT…” and, in your feedback, “none of the above” or “A and C only.”
If you use “all of the above,” make sure it isn’t always the correct answer.
Make sure the answers to assessment questions can be found in the course; if you want learners to use data sheets or other sales tools, indicate that in the instructional text.

What has worked for you when you write assessments?

Training Success

Posted on August 20, 2015 by Seniye Groff

I recently finished a project rolling out a new software coupled with new processes. As always, there were some great insights and learning at the end of the project.

Background:

Learners were dispersed globally and, of course, time and money were important.
The new platform consisted of four new modules that worked with the customer’s business.
Most learners would use one or two of the modules but not all four.
Additionally the new process needed to be shared and include why, what, when, how and where needed to be presented to everyone. Many people were involved with the due diligence so were familiar with the content.
Face-to-face sessions were offered worldwide after a slew of trainers when through train-the-trainer (TTT) sessions.
Documentation was thorough and consisted of process decks, work instructions and quick reference guides for both process and the tools.
Attendance to training was required (yay!).

Reality:

Attendance was surprisingly low, partially I believe because roll out was in August when vacations are at an all-time high. Also the required training was two days long, which was another obstacle because work schedules were already so full. Some attendees felt the class went too fast, while most felt the class was too long. Other feedback stated that the class was too process heavy and not enough tool training. Finally, given that the 700 learners came from many facets of the business, the trainers knew and delivered content on the overall process and tool, but were unable to provide specific business detail to the various departments rolling out the tool. Oh, and the tool was in user acceptance testing (UAT) so it behaved wonky at best during the training sessions.

Future Plan:

As this project matures and future modules evolve and roll out I recommend the following:

A modular approach so that learners only need to attend the sessions that apply to their work. So if they only use one of the four modules in real-life, then they only attend the session on that specific content.
E-learning component so that the learners can learn foundational knowledge in a self-directed format and attend classroom time to work in the tool.
Leader from the respective business group attend the session with their team so that the process-specific questions get answered immediately and demonstrated in the tool.

Performance improvement is an ever-evolving activity. I love the variety and challenge that each project brings; there is no one solution for any project which ensures that creativity and best practices are always required for each and every project!

The Beginning and the End

Posted on March 26, 2015 by Seniye Groff

Similar to the joy of seeing the first spring flower or the first snowy, winter day, the beginning and end of a consulting assignment is most exciting for me. I can set the stage for success or failure by specific actions I take at both critical points in any engagement. The bookends of a project are your client’s impression bank, long after you are onto another assignment. Throughout a consulting assignment, ethics and strong communication skills are keys to a successful engagement. Unless you do what you say you are going to do and communicate along the way, the outcome will not be seen as success in your client’s eyes.

So what can you do to ensure a strong beginning and end to any of your projects?

In the beginning, the client wants to confirm to himself (or even the entire organization) that hiring you was money well-spent and that the time SMEs will spend with you (instead of doing their own job) will be a sound investment. Ultimately, will you solve the problem that you were hired to solve? By rolling up your sleeves, asking good, thought-provoking questions and listening, you can confirm to your audience that you “get it”. Don’t jump to conclusions and presume you have all the answers. Listen to the real problem to determine the root cause of the issue you are asked to solve. On the other hand, you must move fast. Unlike an employee, there is no “on-boarding” time for you; you need to put in the hours to understand the environment, culture, players and issue as quickly as possible. You were hired because you are the expert and you must demonstrate that expertise quickly in the relationship.

Assuming you met your deliverables and solved the problem (and perhaps a few additional issues) how do you end the assignment successfully? I firmly believe that you thank your stakeholder personally–either in person or with a hand-written card. I realize that in this day of email, it is easier to hurriedly spew of a few lines, but I believe it is much more meaningful to do it in person. Secondly, write up a transition plan prior to your departure, highlighting what you have accomplished, where documentation is located and any follow-up activities to ensure what you suggested or created remains active in the organization. Finally, make sure you connect on LinkedIn and even ask for a recommendation, while your success is present in the client’s mind.

What have you done to ensure a strong beginning or end to projects you have worked on?

The Gift of Instructional Design

Posted on December 10, 2014 by Seniye Groff

I recently designed and delivered a session on strategic business planning skills. The topic had been trained for the two prior years but with little success; either the training was too “numbers focused” or too high on the “feelings” barometer. Also, like so many training programs, after the sessions, people failed to take action.

The Performance Problem:
From my initial client meeting I gathered that there was a potential lack of skill, a planning form that was 18 (yes 18 pages!) and little change in behavior after the training due to a lack of follow-up and accountability on the part of the managers.

Solution:
Given the above challenges, I designed a three hour session built on interactivity, sharing of knowledge and accountability through peer-to-peer coaching. First, I worked with the stakeholder to streamline the planning document. And I mean STREAMLINE by turning it into essentially five core pages of information. I felt that possibly some of the reason why the employees were not completing the document was because it was frankly, too daunting. I felt the process was similar to doing year-end taxes and we know how much we love to do those!!

Secondly, I created a lively session that focused on group work. I broke each section of the business plan into discrete components. Participants gathered to discuss a component, including sharing ideas of what they had done in the past. Once the group talked, I brought the discussion to the larger group and more ideas were shared. When the section was completed, participants moved around to work with a different set of people. This in essence got participants to interact with many different people with many different ideas, perspectives and experiences. During the session, it was often difficult to get people to stop talking and sharing (exactly what I wanted to happen)!!

Thirdly, at the end of the session, each person chose a peer to hold accountable and hold them accountable. I felt it was much more likely that participants would take action if someone was going to call them on it.

Results:
All six sessions went great and initial feedback has been VERY positive. It will be interesting to see how 2015 business results are affected by this training. My session design was not by accident; it was a thoughtful process intended on changing performance…and that my friends is the beauty and art of instructional design.

The Art of Feedback

Posted on June 13, 2014 by Seniye Groff

I recently designed and delivered a session on giving and receiving feedback. It is truly remarkable that people cringe at the suggestion of giving feedback. Feedback is a critical skill for success. In sports there is constant feedback: the score, how the ball was hit, whether you returned the serve. In business, feedback is equally as important because it allows you to stay in business, show a profit and create an environment where employees and colleagues are an asset to the business.

Even when feedback is constructive it can be difficult to hear and some people will be reluctant to agree improvement is needed. Yet constructive feedback is a valuable “gift” to help people improve and grow. If feedback is given with care, it is more likely to be truly heard and acted on.

If you focus the feedback on the behavior and not the person, you are likely to maintain the person’s self-esteem. A person should not leave the feedback meeting feeling “beat up” but rather focused on the behavior that needs correction and a plan on how to correct it most effectively.

Remember the 80 / 20 rule and focus on affirming even small successes. Give specific (behavior based) examples of the factors that contributed to the feedback. Make sure the timing is appropriate to give and receive feedback.

For example, “Mike I just listened to your call with Mr. Jones. (Timely) You did a nice job of listening to his complaint in its entirety and then repeating back to him what you thought his issue concerned.” (Behavior based).

It is important to be specific. For example, avoid saying things like, “You need to be more talkative in meetings.” It’s too ambiguous and can be interpreted in a lot of ways.

Say something specific and positive pointed at the task you want accomplished, such as, “You’re smart. I want to hear at least one opinion from you in every meeting we’re in together going forward.”

Specific Steps for Success:

Step 1: State the constructive purpose of your feedback.

State your purpose briefly by indicating what you’d like to cover and why it’s important.

Example: “I have a concern about…”

“I feel I need to let you know…”

“I want to discuss….”

“I have some thoughts about…”

Step 2: Describe specifically what you have observed. Focus on behavior rather than the person. Define the impact.

Have a certain event or action in mind and be able to say when and where it happened, who was involved, and what the results were. Stick to what you personally observed and don’t try to speak for others. Avoid talking vaguely about what the person “always” or “usually” does.

Avoid “You” statements

“You did this” versus “this is what I observed”
“You should not have done that” versus “Here’s how I think…”
“You must be crazy” versus “it is important we talk about this”
“I feel frustrated when you do not get your reports to me on time” versus “I feel frustrated when I do not have the reports on time for the 2:00 meeting” (shows impact of behavior)

Example: “You demonstrate a high degree of confidence when you answer customer questions about registration procedures “, versus, “Your communication skills are good.” Or “You are really great at that.”

Example: “You talked a lot during the staff meeting, which prevented me from getting to some of the main points,” rather than “You talk too much.”

Step 3: Describe your reactions.

Explain the consequences of the other person’s behavior and how you feel about it.

Example: “The team member looked embarrassed and I felt uncomfortable about seeing the episode.” “Shouting at our team members is not acceptable behavior in this department.”

Step 4: Give the other person an opportunity to respond.

Remain silent and meet the other person’s eye, indicating that you are waiting for answer. If the person hesitates to respond, ask an open ended question.

Example: “What do you think?”

“What is your view of this situation?”

“What is your reaction to this?”

“What are your thoughts?”

Step 5: Offer specific suggestions.

Whenever possible make your suggestions helpful by including practical, feasible examples.

Example: “Alison, I sometimes write myself notes or put up signs to remind myself to do something.”

“Sara, rather than telling Matt that you’re not interested in all the details, you might try asking him specific questions about the information you are most interested in.”

Receiving Feedback

Listen without getting hooked. Let the person say their entire thought without interrupting them.
Ask for clarification. Ask for specific examples so that you know exactly what the employee is saying.
Separate your feeling from the facts or specific behavior. It is difficult to hear that we are not perfect. Remember the employee is not saying you are a bad person, just that the particular behavior is not effective for them.
Summarize what you heard. Repeat the feedback to ensure you understood what they were saying.
Ask for advice. A great question to ask is “if you were me, how would you handle this situation?”
Thank the person. Remember, feedback is a gift.

Recent Customer Service Snafu and Lesson

Posted on March 11, 2014March 13, 2014 by Seniye Groff

There are many valuable lessons I have learned from being a parent. Two that stand out in my mind are one, birthdays are very important. So important, they can almost be similar to a national holiday in a child’s mind. And second, kids do not forget (or forgive) when you screw up something related to their birthday. I learned this when I failed to get my daughter a birthday card for her last birthday. I heard about it for this past year repeatedly. (I am proud to say not only did I not forget this year, but I actually hand made the card!)

Given these lessons, I have to tell you about a recent experience in my local Portland ice cream store (to remain nameless). I typically bake cupcakes for my daughter’s birthday but this year wanted to do something a little different for our family celebration. So, last Sunday, I called the ice cream store to order an ice cream cake. My soon-to-be 12 year old and I poured over the ice cream store’s website looking for the perfect cake. My daughter decided on the watermelon cake with chocolate ice cream and vanilla cake. When I called on Sunday evening, the young man told me he would have to call me back because they were busy. I gave him my number. Twenty minutes later he called me back. I told him the cake we wanted and he asked for how many people. I told him eight to ten. He said fine and that it would be ready to pick up the following Saturday.

On Saturday, my husband went to pick up the cake. He was immediately informed that there had been a mix up and that the young man who took my order did not realize there was no such thing as a cake for eight to ten people. The employee stated they tried to call but could not find the number. And so since they could not get a hold of us, they “chose to do nothing.” Nothing???? Really?!! If the employee stopped and thought about it for one moment, he would have realized #1. This is a BIRTHDAY cake order and #2. It is for eight to ten people (which means there is some type of get together involved) and #3. If the person picking up this cake is planning on having this cake, what other options would they have with such late notice and #4. Again, this was a BIRTHDAY cake for a CHILD.

My husband was offered another cake out of the case. There were not a lot of choices and frankly, we had nine people to feed in less than an hour so we did not have any other options. My husband chose a cake which was not even close in style or flavor to what my daughter wanted.

And the end result: my daughter CRIED when she saw that the cake she had been waiting all week for, was indeed not what she had waited all week for! My husband and I wondered why better decisions could not have been made. For example, make the watermelon cake for the correct size the ice cream store makes it for and call it good.

Of course, I called the manager the next day to explain what happened and to ask WHY? I was told that not everyone is trained in cake details and probably the person I worked with did not have the cake training. Well, no doubt about that! Also the manager explained that the person I worked with was young and young people do not always make great decisions. Hmmmm! That explanation really makes you wonder.

I propose a few solutions to ensure great customer service:

1. Train everyone on the cake details. If that is not possible, how about ensuring a customer ordering a cake, is given to the “cake expert” in the store.

2. Create a quick reference guide about the cake ordering process. A checklist could be included to make sure all details are covered.

3. Offer some training for employees on how to troubleshoot and solve problems. It could be structured almost like a FAQ (frequently asked questions) document.

These are just three quick ideas. What other ideas to YOU have?

One thing for sure, I am back to making cupcakes next year, when my daughter turns 13!

Physical Versus Mental Parenting

Posted on January 20, 2014 by Seniye Groff

I remember the days of sheer physical exhaustion from chasing after my kids; always worrying if they were going to tumble down the stairs or get into some type of poison underneath the kitchen sink. Now that the kids are six and 12, there has been a noticeable shift in my worries–especially for the 12 year old. I no longer am physically exhausted from playing hide and seek and chasing after them down the block. Instead, I now am in the role of “detective”. I have to review my older daughter’s website history, her emails, her texts and various accounts like Instagram and Snapchat. There seems to be all sorts of dangers lurking and an innocent child has no idea of the possible outcomes.

When I chat with other parents and tell them of my trials and tribulations, most are shocked. Their first reaction is “How do you know?” and then “How do you check?” Ignorance is not bliss here, people. As a parent, you must know what your child is doing online. Parents should know every place their child goes online and should have every user name and password to every account. I am currently creating a job aid for parents so that they know how to check the most commonly sites and apps that 11-13 year-olds are using. Of course, things move rapidly, so fully expect to update the document regularly.

What things do you need to know how to do?
What sites does your child visit regularly?
Have you implemented internet “rules” that your child must follow?
Does your child have an email address?
Does your child text?

Let me know what your experience is with your child and the online environment. Also let me know what would be helpful to know so that I can consider putting it on my document.

← Back

Thank you for your response. ✨

It’s the Way You Say It: Becoming Articulate, Well-spoken, and Clear

Posted on December 9, 2013March 13, 2014 by Seniye Groff

t’s the Way You Say It is a goldmine of usable information to improve your verbal and nonverbal skills. Carol A. Fleming is a speech and language pathologist who shares her techniques for voice and communication. Chapter one begins with an assessment. Chapter two covers nine vocal challenges while chapter three reviews vocal techniques. Chapters four and five review what to say coupled with nonverbal body language. Chapter six includes business communication situations. Fleming shares many client stories by discussing the initial communication issue, what she recommended and what the client did to overcome the challenge. Fleming does not offer any “quick fix” remedies, but instead gives an action plan expecting the client to practice to improve the skill.

” People easily make the leap from the sound of your voice to the nature of your personality.”

It’s the Way You Say It has so much valuable information, you will want to read this book over and over, highlight the meaningful areas and practice the activities that Fleming suggests. You can address a single issue or your entire communication pattern with this incredible resource. Buy it, use it, practice it and watch your communication skills soar!

Driving Loyalty: Turning Every Customer and Employee into a Raving Fan for Your Brand

Posted on December 9, 2013March 13, 2014 by Seniye Groff

Kirk Kazanjian does not offer a magic bullet in his book Driving Loyalty. His advice is straightforward: take care of your employees and your customers and your business will succeed. This book has a little bit for everyone. New managers will gain lots of new insight on how to reward and retain employees, while seasoned managers will learn how to implement a company mission and vision to reinforce the focus on employees and customers. The book details effective merger tactics, as well as using technology to delight customers and employees. Kazanjian also covers growth, partnership and sustainability. There really is a ton of information for a mere 272 pages!

“Never underestimate the importance of the role those on your team play in delivering on your brand promise.”

Driving Loyalty is easy to read given its vast amount of usable information and key points summary at the end of each chapter. Throughout the chapters there are tables, numbered lists and graphics to highlight the most vital ideas from each chapter. Highlight the significant concepts, test some of the ideas and implement what works for your business. I am pretty sure you will notice a change in both your employees and customers.

seniyegroff

Eclectic musings from a wanderer, reader, cook and mom

Performance Improvement