What Is Assessment?
While there are many variations in the definition of assessment, for the purposes of this paper we will build on the one from the Association of Assessment of Higher Education, AAHE Bulletin, November 1995. Assessment is an ongoing process aimed at understanding and improving student learning. It involves:
- Making our expectations explicit and public;
- Setting appropriate criteria and high standards for learning quality;
- Analyzing and interpreting evidence to determine how well performance matches those expectations and standards.
Why Do Assessment?
The research and information about assessment is almost boundless which demonstrates the profound interest in this subject. The 21st century with its knowledge driven economy now, more than ever, demands that institutions produce a quality education for the students they serve. One of the ways to demonstrate the quality of that education is to measure it authentically and communicate the results effectively. This is where assessments become key. Effective assessment can document the progress of students, faculty and institutions based on measured outcomes.
Assessment Research, Principles & Trends
Where Do I Begin?
This paper will expose you to the most pertinent and timely information and trends now emerging, in hopes that these will be immediately helpful in your quest to design, develop improve and/or extend the assessment process in your classroom, division and college.
This paper presents 9 Principles of Good Practices for Assessing Student Learning and 5 Assessment Trends for Business Educators and Computer Information Systems Instructors. Embedded in these principles and trends are Web links for you to further explore, in order to gain knowledge and ideas for creating your own assessment approach.
This interactive white paper is intended to serve as both a resource paper and tool-kit. We recommend that you read the paper the first time, through to its entirety. Then, go back and explore each of the links at your leisure and as appropriate for your assessment needs.
The Web is a wonderful and powerful tool for gathering resources and sharing ideas. The links presented here have been collected from industry specialist and educators like you. It by no means is a complete collection of the wonderful concepts and applications available on the Web. Fortunately, and unfortunately, the Web offers such a vast amount to support material and research examples, daily, and to be able to include every item that addresses our challenges and concerns would be a paper that could never be complete.
However, please think of this document as ?a living document? for you to add to as you grow in your own assessment strategies and methodologies.
Doing Assessment As If Learning Matters Most.
Thomas A. Angelo wrote an article (AAHE Bulletin, May 1999) where its title expresses the most critical and challenging task of business educators today, that is, Doing Assessment As If Learning Matters Most. In his paper Angelo states the following:
If learning really matters most, then our assessment practices should help students develop the skills, dispositions, and knowledge needed to:
- Engage actively ? intellectually and emotionally ? in their academic work.
- Set and maintain realistically high, personally meaningful expectations and goals.
- Provide, receive, and make use of regular, timely, specific feedback.
- Become explicitly aware of their values, beliefs, preconceptions, and prior learning, and be willing to unlearn when necessary.
- Work in ways that recognize (and stretch) their present learning styles or preferences and levels of development.
- Seek and find connections to and real-world applications of what they?re learning.
- Understand and value the criteria, standards, and methods by which they are assessed and evaluated.
- Work regularly and productively with academic staff.
- Work regularly and productively with other students.
- Invest as much engaged time and high-quality effort as possible in academic work.
The Web resources showcased throughout this paper allow us to illustrate Angelo?s guidelines presented above. But these examples, and any list of assessment guidelines, will only be useful to the extent that we, the assessment activists, first establish the fundamentals as stated by Angelo.
To achieve transformation in higher learning, we must develop shared trust, a transformative vision of goals worth working toward, and shared language and concepts equal to the challenge. If we plan and conduct our assessment projects at every step as if learning matters most ? and not just student learning, but ours as well ? then the distance between means and ends will be reduced and our chances of success increased.
To see Angelo?s complete article, click on www.aahe.org/bulletin/angelomay99.htm.
9 Principles of Good Practices
for Assessing Student Learning
There are many websites, books, articles and seminars loaded with information about assessment. A good place to start is the AAHE, American Association of Higher Education Assessment Forum www.aahe.org/assessment/. This AAHE Assessment Forum is considered the primary national organization that connects and supports higher education assessment. At this site there are literally hundreds of links to articles, books and websites all carefully categorized.
One document that repeatedly appeared when looking for assessment information was the AAHE?s, 9 Principles of Good Practices for Assessing Student Learning. A number of leading experts on assessment, including Patricia Cross and Peter Ewell, wrote this piece. This is where we will begin. We will be studying one principle at a time. You can see the two-page paper in its entirety by clicking on the link www.aahe.org/principl.htm.
The assessment of student learning begins with educational values. Assessment is not an end in itself but a vehicle for educational improvement. Its effective practice, then, begins with and enacts a vision of the kinds of learning we most value for students and strive to help them achieve. Educational values should drive not only what we choose to assess but also how we do so. Where questions about educational mission and values are skipped over, assessment threatens to be an exercise in measuring what's easy, rather than a process of improving what we really care about.
According to this first principle teachers need to focus on their goals and values for the classes they teach. You must, as Stephen Covey loves to say, ?begin with the end in mind.? To help you begin to identify these goals and values, complete the questionnaire entitled Teaching Goals Inventory www.siue.edu/~deder/assess/cats/tchgoals.html.
After reading the instructions, you may want to print out the inventory and complete it. When finished, return to the site and click on the self-scoring worksheet at the bottom of the page.
- Were you able to identify your goals?
- How will you use this information?
Assessment is most effective when it reflects an understanding of learning as multidimensional, integrated, and revealed in performance over time. Learning is a complex process. It entails not only what students know but what they can do with what they know; it involves not only knowledge and abilities but values, attitudes, and habits of mind that affect both academic success and performance beyond the classroom. Assessment should reflect these understandings by employing a diverse array of methods, including those that call for actual performance, using them over time so as to reveal change, growth, and increasing degrees of integration. Such an approach aims for a more complete and accurate picture of learning, and therefore firmer basis for improving our students' educational experience.
If you would like a quick review of the levels of learning, click on the link below to take you to Blooms Taxonomy for Learning. The chart will remind you of the different learning domains and how to measure them www.coun.uvic.ca/learn/program/hndouts/bloom.html.
- Are you creating assessments that measure the simplest, knowledge, to the most complex, evaluation?
If we know that learning is multidimensional and occurs on many levels, then we must create assessments that do the same. Multiple-choice tests can only measure certain aspects of learning. There are a host of assessments that will measure a variety of levels of learning.
The next challenge will be to match your course goals with various classroom assessment techniques (CAT). Click on the link and complete the Matching Goals to CAT www.wcer.wisc.edu/nise/CL1/flag/goals/goals.htm.
Now that you have some ideas about what techniques will work for the objectives you are trying to reach, go to the site below and scroll down to the CATS At a Glance. Click on each assessment strategy and you will get a brief description. On the left hand side of the screen is an in-depth write- up of each technique, available by clicking on the link. Here you will find an extensive definition, actual examples and insight from instructors who use this strategy www.wcer.wisc.edu/nise/CL1/flag/cat/cat.htm.
- Were you able to match your goals with appropriate assessment strategies?
Assessment works best when the programs it seeks to improve have clear, explicitly stated purposes. Assessment is a goal-oriented process. It entails comparing educational performance with educational purposes and expectations -- those derived from the institution's mission, from faculty intentions in program and course design, and from knowledge of students' own goals. Where program purposes lack specificity or agreement, assessment as a process pushes a campus toward clarity about where to aim and what standards to apply; assessment also prompts attention to where and how program goals will be taught and learned. Clear, shared, implementable goals are the cornerstone for assessment that is focused and useful.
Alverno College is a four-year, liberal arts college for women, located in Milwaukee. Their faculty came up with an ability-based education program that applies to all students and all courses taught at the college. Check this out and see what you think. Be sure to scroll down the page and check out the assessment section www.alverno.edu/academics/ac_curriculum.shtml.
- Does your class, department, division and college have clear and concise goals?
- What do you think of Alverno?s approach?
Assessment requires attention to outcomes but also and equally to the experiences that lead to those outcomes. Information about outcomes is of high importance; where students "end up" matters greatly. But to improve outcomes, we need to know about student experience along the way -- about the curricula, teaching, and kind of student effort that lead to particular outcomes. Assessment can help us understand which students learn best under what conditions; with such knowledge comes the capacity to improve the whole of their learning.
Most instructors construct assessments based on their own learning style. Implementing only one style of assessment will not meet or measure student success. Click on this link to complete a Learning Styles Questionnaire
- What?s your style?
- Are you teaching and assessing to a variety of learning styles?
- How can you move beyond your style to construct a variety of assessments?
Take a quick look at this article entitled Learning Styles and Strategies www2.ncsu.edu/unity/lockers/users/f/felder/public/ILSdir/styles.htm.
- What experiences lead to successful outcomes?
- How can you create assessments that serve all learning styles and more successful outcomes?
Assessment works best when it is ongoing not episodic. Assessment is a process whose power is cumulative. Though isolated, "one-shot" assessment can be better than none, improvement is best fostered when assessment entails a linked series of activities undertaken over time. This may mean tracking the process of individual students, or of cohorts of students; it may mean collecting the same examples of student performance or using the same instrument semester after semester. The point is to monitor progress toward intended goals in a spirit of continuous improvement. Along the way, the assessment process itself should be evaluated and refined in light of emerging insights.
There are a number of assessments that you can use on a daily or weekly schedule that will provide you with a great deal of information about how and what your students are doing. Here is one called Pause Procedure. Read about it and then try it out www.wcer.wisc.edu/nise/CL1/CL/doingcl/pause.htm.
Here are three more quick assessment strategies for you to try. Check out the minute paper, the muddiest point, and the one sentence summary cstl.syr.edu/cstl/t-l/cls_asmt.htm.
- Which one will you try tomorrow?
Check out this information and take a look at the chart defining a variety of assessment techniques www.ntlf.com/html/lib/bib/assess.htm.
- Have you tried a chain note?
- How frequently are you assessing your students?
- How are you using this assessment information to modify your course?
Assessment fosters wider improvement when representatives from across the educational community are involved. Student learning is a campus-wide responsibility, and assessment is a way of enacting that responsibility. Thus, while assessment efforts may start small, the aim over time is to involve people from across the educational community. Faculty play an especially important role, but assessment's questions can't be fully addressed without participation by student-affairs educators, librarians, administrators, and students. Assessment may also involve individuals from beyond the campus (alumni/ae, trustees, employers) whose experience can enrich the sense of appropriate aims and standards for learning. Thus understood, assessment is not a task for small groups of experts but a collaborative activity; its aim is wider, better-informed attention to student learning by all parties with a stake in its improvement.
Check out how Maricopa College supports assessment www.mcli.dist.maricopa.edu/ae/. Maricopa works hard to get everyone in the learning community involved in the assessment process.
- What college wide support are you getting?
Assessment makes a difference when it begins with issues of use and illuminates questions that people really care about. Assessment recognizes the value of information in the process of improvement. But to be useful, information must be connected to issues or questions that people really care about. This implies assessment approaches that produce evidence that relevant parties will find credible, suggestive, and applicable to decisions that need to be made. It means thinking in advance about how the information will be used, and by whom. The point of assessment is not to gather data and return "results"; it is a process that starts with the questions of decision-makers, that involves them in the gathering and interpreting of data, and that informs and helps guide continuous improvement.
Ruth Stiehl in her book, The Outcomes Primer, asks a question that can really help drive your assessment decisions. ?What do my students need to be able to do ?out there? (in the rest of life) that we are responsible for in this classroom?? Once you identify the outcomes, the assessment is merely finding appropriate ways for students to demonstrate their capability to meet those outcomes, says Stiehl. Go to this article by Stiehl and check out the chart for 21st Century Outcomes www.ctt.bc.ca/lo/Cdmodels.html.
Now review this article on performance assessments. It is short and to the point
- Are you using performance based or authentic assessments? If not, why not?
Assessment is most likely to lead to improvement when it is part of a larger set of conditions that promote change. Assessment alone changes little. Its greatest contribution comes on campuses where the quality of teaching and learning is visibly valued and worked at. On such campuses, the push to improve educational performance is a visible and primary goal of leadership; improving the quality of undergraduate education is central to the institution's planning, budgeting, and personnel decisions. On such campuses, information about learning outcomes is seen as an integral part of decision making, and avidly sought.
Here is a university that measures reports and is committed to quality assessment. Check it out and see if you agree www.washington.edu/oea/asesprin.htm.
Revisit this article to find the 10 Guidelines for Assessing as if Learning Matters Most www.aahe.org/bulletin/angelomay99.htm.
- Does learning matter most in your classroom and on your campus?
- Does your campus culture value learning and make decisions based on learning outcomes?
Through assessment, educators meet responsibilities to students and to the public. There is a compelling public stake in education. As educators, we have a responsibility to the publics that support or depend on us to provide information about the ways in which our students meet goals and expectations. But that responsibility goes beyond the reporting of such information; our deeper obligation -- to ourselves, our students, and society -- is to improve. Those to whom educators are accountable have a corresponding obligation to support such attempts at improvement.
Here is an interesting program developed by and for many stakeholders. Take a look and see what you think www.evergreen.edu/user/washcntr/lcdir.shtm.
- How can you get more stakeholders involved in the assessment process?
The next step is to review assessment information that directly pertains to Business Educators in general and more specifically those in California Community Colleges. Let?s look at 5 Assessment Trends for Business Educators and Computer Information Systems Instructors. Research of the literature suggests that these 5 trends are some of the most critical challenges and opportunities with respect to assessment in business education.
5 Assessment Trends for Business Educators and Computer Information Systems Instructors
Evaluating, assessment and reporting of information is important in the business community. Businesses of all kinds are developing and following specific standards to produce quality products. They measure and evaluate their processes using a set of assessments often carried out as audits. Here are links to three of the tools businesses use to measure how they are doing. Check out each one:
Baldridge Award www.quality.nist.gov/
Shingo Prize www.shingoprize.org/shingo/index.html
- How do these assessments help keep business focused on their goals?
- Why would business put so much time, money and energy in these assessment processes?
- Why is this of interest to business education instructors?
Certification is a growing assessment tool being used for information technology workers.
There are well over 350 information technology certifications. According to a report in Information Technology Association of America, ? Employers may list certification as a plus, but it?s usually not a requirement.? Check out this site for more information on certification, academic standards and assessments to enhance learning and develop IT skills www.edc.org/EWIT/curr.htm.
In this article find out some of the reasons for certifications. Certificate Quality in the Knowledge Economy gives some reasons why employers like the idea of certification
- Are the students you teach required or encouraged to get certification of skills?
- What is the value of certification?
- How do you keep abreast of the changing certification requirements?
Authentic and Performance Based Assessments are becoming mandatory for business education courses. The same theme keeps coming to the forefront in assessment.
- What can the student really do?
- Is the student able to transfer the information or skill learned in class to a real world setting?
Businesses want students to demonstrate their skills and abilities, before they hire a candidate they are conducting behavioral interviews, evaluating student portfolios and requiring a battery of pre-employment tests. Read some more about learning outcomes in the paper below. If your time is limited read the ?short answer? section toward the end of the document www.ctt.bc.ca/lo/sowhatsa.html.
For performance assessments try this ericae.net/pare/getvn.asp?v=5&n=7.
Want to know more about behavioral interviews? Try this resource content.monster.com/military/articles/behaviorbased/.
This site explains and provides examples of student portfolios
- How do you feel about learning outcomes?
- What are you using in your courses?
Rubrics are being used to help construct and measure assessment information. Rubrics can guide a student in a self-evaluation process, be used as a tool for teaching processes and standards, and they are an easy way to focus on what it is an instructor wants students to get from the class. Here is more information on rubrics edweb.sdsu.edu/webquest/rubrics/weblessons.htm and
Want to build a rubric? Here you can sample some rubrics and get more information about the variety of rubrics and how to create them
- How can rubrics be useful in designing assessments?
There is a continued necessity for lifelong learning and assessment for today?s knowledge worker. Gathering and using skills is not a one-time phenomenon but an ongoing process that will continue to follow employees throughout their careers. Want to keep abreast of changes? What does the knowledge worker continue to learn and grow in the new economy?
Higher education is the only business that has a ceremony for firing its customers. The current model of learning and training?will have to change to one that keeps its learners engaged over a long period of time?.The financial survival of educational institutions and the growing need for continuous, lifelong learning demand such a change. Rather than offer learning that has an end point transform learning into something continuous?Educational institutions that survive will move from the Industrial Age ?event? model to a model that turns students into members of a network-a network that keeps them engaged over the course of their life. Elliott Masie, The MASIE Center www.masie.com.
Here is the text from a speech made June 2001 by Alan Greenspan, The Growing Need for Skills in the 21st Century. What does Greenspan say about lifelong learning?
Check out this article on the learning organization
- Do you agree or disagree with the quote by Elliott Masie?
- Why or why not?
The 9 Principles of Good Practices for Assessing Student Learning and the 5 Assessment Trends for Business Educators and Computer Information Systems Instructors can be summarized as follows:
The assessment of student learning begins with educational values.
Assessment is most effective when it reflects an understanding of learning as multidimensional, integrated, and revealed in performance over time.
Assessment works best when the programs it seeks to improve have clear, explicitly stated purposes.
Assessment requires attention to outcomes but also and equally to the experiences that lead to those outcomes.
Assessment works best when it is ongoing not episodic.
Assessment fosters wider improvement when representatives from across the educational community are involved.
Assessment makes a difference when it begins with issues of use and illuminates questions that people really care about.
Assessment is most likely to lead to improvement when it is part of a larger set of conditions that promote change.
Through assessment, educators meet responsibilities to students and to the public.
Evaluating, assessment and reporting of information is important in the business community.
Certification is a growing assessment tool being used for information technology workers.
Authentic and Performance Based Assessments are becoming mandatory for business education courses.
Rubrics are being used to help construct and measure assessment information.
There is a continued necessity for lifelong learning and assessment for today?s knowledge worker.
Confused about any of the terms in assessment? Here is an on-line assessment glossary that will help. cresst96.cse.ucla.edu/CRESST/pages/glossary.htm.
Stay on the Journey.
Keep Asking Where are We Going and Why?
As we move into the 21st Century, business educators, probably more than educators from other curricula areas, will see profound changes. In order to keep pace with the rapid changes and serve the students who depend on you and the direction you provide, ask yourself the question so aptly put by Ruth Stiehl, ?What do my students need to be able to do ?out there? (in the rest of life) that we are responsible for in this classroom?? If you keep asking that question and listening to the answers, you will be well on your way to doing good assessment!
Technique, methodology, the process you?ll need to follow?this will come to you. Just make sure your aim is good, open yourself to the unexpected and proceed. You don?t have to know how you?re going to get there, but you must know where you want to go. Price Pritchett, The Quantum Leap Strategy.
Before You Go
Please feel free to provide us any feedback on this interactive white paper. Your input is appreciated!
For questions and comments regarding this interactive white paper, please contact: Joyce Arntson:
The following Websites have been referenced in this paper, and in the order as they appear here:
AAHE Assessment Forum
AAHE?s 9 Principles of Good Practices for Assessing Student Learning.
Goals of Assessment
Blooms Taxonomy for Learning
Matching Learning Goals to CAT
Measuring Learning Outcomes
Assessment and the Business Community
Performance Assessments and the Business Community
National Business Education Association (NBEA)
California Business Education Association (CBEA)
TNT & BESAC Websites
At the National Business Education Association (NBEA) site you will find the National Standards for Business Education based on a comprehensive model which includes 12 content areas. These 12 standards areas include many of the subjects included in the discipline of business education. These include: accounting, business law, career development, communications, computation, economics and personal finance, entrepreneurship, information systems, international business, interrelationships of business functions, management, and marketing www.nbea.org/curfbes.html. Check out the achievement standards and the list of additional resources to help you find assessment ideas.
The California Business Education Association (CBEA) represents professionals involved in education for and about business at all levels of public and private institutions. Try out their links and resources www.cbeaonline.org/. What tools and resources are you using to stay updated about Business Education in general and assessment more specifically?
The US Department of Education, a comprehensive education resource covers access, grants, programs and teaching materials. Check headlines, studies and research findings www.ed.gov/.
ASTD is the world's leading association of workplace learning and performance professionals, forming a world-class community of practice. ASTD's 70,000 members come from more than 100 countries and 15,000 organizations.
The Resource Centers of Blackboard.com allow you to access thousands of full-text journal articles, augment your course with robust educational content from hundreds of publishers, and review discipline specific news, articles, and tutorials. There is an Academic Resource Center, Instructor Center, Student Center, Blackboard Communities, Training Center and a Course Content Center.
Brandon Hall: www.brandon-hall.com
Brandon Hall provides independent, objective information about using technology for learning to help you make the right decisions for your organization. We keep you well-informed on trends, best practices, tools and vendors.
Centra Software, Inc.: www.centra.com
Centra created of Centra One, an Internet-based platform for delivering live instructor-led learning or virtual events.
Cisco System, Inc.: www.cisco.com
Education is the great equalizer in life. In order to properly prepare our children for the jobs of the 21st Century, we need fundamental changes in our education system. Government leaders, teachers, parents, and businesses need to embrace accountability and competition in our schools if we are ever going to improve the current situation. John Chambers, CEO & President, Cisco Systems, Inc.
Check out Cisco?s commitment to education and education in the Internet economy www.cisco.com/warp/public/779/edu/commitment/edu_internet_economy/
Also explore Cisco E-Learning www.cisco.com/warp/public/10/wwtraining/elearning/
Generation 21: www.gen21.com
Creators of the Total Knowledge Management (TKM) system, a Learning Content Management System (LCMS).
The Masie Center: www.masie.com
The world of Learning is dramatically changing in 2001. We are tracking significant increases in learning that is targeted at customers and the "supply chain" as well as employee development. In fact, The MASIE Center believes that in three to four years, there will as much customer and supplier e-Learning offered as employee oriented programs. In addition, the economy is presenting both challenges and opportunities to organizations developing Learning Strategies. Elliott Masie, The MASIE Center & TechLearn Conferences.
Question Mark Perception: www.questionmark.com/perception/
Creators of the assessment tool Perception.
The Training Place: www.trainingplace.com
The Training Place focuses on motivating learners and improving performance.
The Training Place offers customized content, learning and content management technology, consulting services, hosting, and learning orientations research to deliver personalized e-learning solutions.
On-Line Learning Magazines
Books to Augment Your Assessment Learning
Alessi, S.M. & Trollip S.R. (2001) Multimedia for Learning, Methods and Development. 3rd ed. Boston, MA: Allyn and Bacon Publishers.
Horton, W. (2000) Designing Web-Based Training - How to Teach Anyone Anything Anywhere Anytime. New York, NY: John Wiley & Sons.
Kirkpartick, D.L. (1998) Evaluating Training Programs. 2nd ed. San Francisco, CA: Berrett-Koehler Publishers
Worthen, B.R., White, K.R., Fan, X., & Sudweeks, R.R. (1999) Measurement and Assessment in Schools. 2nd ed. New York, NY: Longman Publishers.