evaluation

Select an item by clicking its checkbox

Persons new to the office of the Dean may soon discover the need to acquire a new set of skills to effectively carry out the job. Those skills range from supervision, pastoral care (yes, more than you imagined!), educational administrative planning, curriculum design and planning, political acumen, budgeting and financial ...

Reviewed by: Israel Galindo
Date Reviewed: 2018-07-18
Arguably, due to suspicion, skepticism, and the entrenchment of classroom teaching practices, the most scrutinized, assessed, and evaluated pedagogy in theological schools in the last two decades has been online learning (“distance education”). Fortunately, the skepticism that fueled a demand for more rigorous evaluation of student learning in the online environment has yielded hard evidence that online learning is as effective, and sometimes more effective, than traditional classroom teaching. The ...

Arguably, due to suspicion, skepticism, and the entrenchment of classroom teaching practices, the most scrutinized, assessed, and evaluated pedagogy in theological schools in the last two decades has been online learning (“distance education”). Fortunately, the skepticism that fueled a demand for more rigorous evaluation of student learning in the online environment has yielded hard evidence that online learning is as effective, and sometimes more effective, than traditional classroom teaching.

The goal of this work by Catalano, an associate professor of teaching and learning at Hofstra University, is “to assist researchers of distance education, both novice and expert, in finding well-developed and value measures to suit their research questions” (2). The focus of the book is “on properly validated measures, thereby relieving the researcher from having to develop his or her own instrument” (2).    

Catelano provides a concise guide to seventy selected assessment instruments (surveys, scales, and methods) that have been used to evaluate the effectiveness of online learning programs. The instruments can be applied to evaluate key metrics and pedagogical, as well as programmatic, elements such as: student engagement, student and faculty satisfaction, information retention, self-efficacy, student and teacher readiness, the online learning environment, cooperative learning, competencies for online learning, student attitudes, student retention and attrition, critical thinking, and achievement. Several instruments focus on specific pedagogical approaches such as constructivism and andragogy.

Five chapters provide nine categories of evaluations. Specifically, Chapter 1 provides eleven instruments on student engagement and satisfaction. Chapter 2 contains sixteen instruments on student readiness to learn and self-efficacy. Chapter 3 offers a selection of seventeen instruments focusing on the online teaching and learning environment. Chapter 4 provides twelve instruments on student learning behaviors. Chapter 5 contains five instruments to assess student achievement, retention, and attrition. Obviously, there are enough offerings in these chapters to meet just about any evaluation need an assessor may have. Each chapter includes a brief overview of each instrument covered. Each instrument entry includes a summary of the tool, a description of how the measure is used, and a description of its development and validity.        

Some of the instruments described are complete tools (for example, a fully developed satisfaction questionnaire, a complete survey on student evaluation of online web-based instruction, and one on the online learning environment. Other instruments are described with information about their sources should an assessor wish to use them.         

In the Introduction Catalano provides helpful background on the criteria for inclusion of the instruments selected for the compendium. The author describes the search process used in identifying the instruments, including applying the criteria of ensuring reliability and validity.

For those needing to continue to assess online learning through disciplined and rigorous evaluations, whether as part of formative curricular assessment or to satisfy accreditation requirements, Amy Catalano’s brief compendium of assessment instruments, scales, and approaches is a handy and accessible tool. Indeed, it should be on every evaluator’s desk.

Cover image

Learning, Teaching and Assessment in Higher Education: Global Perspectives

Book-Review
Brown, Sally
2015
Palgrave Macmillan Springer Nature
icon
Tags: assessment   |   diversity   |   evaluation   |   student learning outcomes

Reviewed by: Joanne Maguire Robinson
Date Reviewed: 2016-02-04
This global study of teaching, learning, and assessment goes beyond the typical single country context to extend to good pedagogical practice on six continents.The argument of each chapter is supported by examples of good practice accounts from around the world, a collection that forms a refreshing change from a generic case study model. Although the examples lean most heavily toward English-speaking institutions and classrooms, there is significant attention paid ...

This global study of teaching, learning, and assessment goes beyond the typical single country context to extend to good pedagogical practice on six continents.The argument of each chapter is supported by examples of good practice accounts from around the world, a collection that forms a refreshing change from a generic case study model. Although the examples lean most heavily toward English-speaking institutions and classrooms, there is significant attention paid to various elements of pedagogy from many cultural perspectives. The author takes great care throughout the book to consider all elements of the changing higher education landscape, including the changing student body, changes in technology, and changes in expectations about the goals of higher education and graduate employability.

The twelve chapters of this book take a comprehensive view of diversity in higher education teaching, learning, and assessment. The book opens with a consideration of cultural mores and assumptions that directly affect interactions in higher education. Brown looks both to the past and the future in subsequent chapters, noting pedagogical traditions and innovations in the context of today’s higher education landscape. She argues that we must adjust to a technology-rich world that necessitates more focus on “learning how and learning why than on learning what” (21). Her view of technology in the balance as both a distraction and a valuable addition to certain aspects of both teaching and learning is refreshing, as so many books either glorify or decry technology in classrooms and society. This ability to see things in the balance is one of the greatest strengths of this book: rather than arguing for a “best” way to teaching or learning or assessment, this book offers multiple possibilities from multiple contexts and thus leaves much to the reader to judge based on his or her context and constraints.

Brown identifies a few global trends. The most widespread seems to be a move from transmissive to transformative education. She notes that there is a “movement from perceiving the university teacher as an all-knowing, unchallengeable authority figure” (27) and parallel a movement by institutions and disciplines toward encouraging learning outside of the lecture hall. Another trend is teaching toward the multiple literacies expected of a twenty-first century graduate, looking well beyond academic literacy to digital, assessment, and interpersonal literacies (88). Finally, Brown notes that all education needs to think of itself as taking place in a global environment. This book is itself a fine way to encourage broader thinking about pedagogical contexts in higher education: our students are shortchanged when we privilege our own pedagogical traditions and ignore the broader, global context of higher education.

The strengths of this book are many. For instance, the author provides substantive and exhaustive bulleted lists in each chapter, a diverse set of highlighted good practice accounts, and a full chapter devoted to higher education teacher development. The book is easy to navigate, written in clear prose, and at once expansive but grounded in particulars. I would recommend this book highly to teachers and administrators in higher education across the disciplines.