Crafting Rubrics and Resources with Generative A.I.

by Nate Pindell, Instructional Designer, Center for Transformative Teaching

When generative artificial intelligence (A.I.) first made its splash in higher education the priority topic on instructor’s minds was academic integrity. How might students be able to use this tool to circumvent academic integrity? The concern driven by reports of students using the A.I. in assignments from discussion posts to final papers. As the ripples of the impact are settling, and with it best practices in assessment creation and grading navigating the agitated waters, the conversations have been focused on the use of A.I. in all ways to assist, not just students who may be using it for nefarious reasons, but for individuals looking to save time in time consuming processes.

Rubrics are a best practice for grading. A well-designed rubric allows students to be able to grade themselves as they progress through the assessment. To clearly see the finish line that instructors want them to cross. Rubrics also save time in grading and make grading fairer. I myself have had many evenings grading a stack of papers only to find myself being less lenient with students towards the end than I was at the beginning. Or teaching assistants not grading homogenously because to them the standards were different in their mind’s eye. No matter how you slice it, parameters identified and detailed in rubrics make the lives of everyone involved more manageable.

But the problem lies in the time it takes to create the rubrics. That while the payoff in time saved outweighs the time put-in, many individuals do not have that “time equity” to invest. Also just putting criteria and descriptions in rows and columns does not a good rubric make. Then, on top of all of time commitment, Canvas rubrics can be rather difficult as the ability to edit the rubrics are limited. These difficulties are to blame why many instructors, who recognize the net benefit that they and their students will reap, are often deterred from making rubrics for as many assessments as they can.

But what if there was a way to create comprehensive rubrics in a fraction of the time?

Enter generative A.I.

Knowledge is Power – Informed Queries Create Better A.I. Output

Before we begin the foray into how to use generative A.I. to create rubrics, it is important to address how the user has all the power in the user/A.I. relationship. I recently heard the analogy of the user being the conductor while the A.I. is the orchestra. If I were to put you in front of 150 person orchestra and ask you to conduct a playing of Beethoven’s Fifth could you do so? The answer is probably no. Knowledge of music theory, the instruments, and the movements the conductor uses to relay information to the musicians is needed for a successful concert. While this is not a one-to-one comparison for users and A.I. it does put into perspective that the more informed you are about a topic the better you can conduct the A.I. to create what you want.

If you are reading this article because you want to use generative A.I. to create rubrics but do not know the types of rubrics that exist, their parts, or how to use them it is strongly recommended that you visit this resource we have put together on the Center for Transformative Teaching website about rubrics. You can also contact an instructional designer assigned to your college or department for a one-on-one consultation if that is your preference!

If you are already familiar with rubrics, or looking to see what the end result looks like before you dive in, then read on!

Crafting Rubrics with ChatGPT

For this demonstration I used ChatGPT. There are other generative A.I.s that exist. But this one has a free option and is the most widely used. So much so that “ChatGPT” is becoming the term for generative A.I. Much like “Band-Aid” is for adhesive bandages or “Zoom” is for video calls and conferences.

Using Generative A.I.

You may be familiar with using ChatGPT and other generative A.I. But to be sure I am going to go over a few pointers.

• Use as simple language as you can/want to. These A.I. do not “think” or “understand” like you may be persuaded to believe. While this A.I. will create amazing rubrics for you, it does not “understand” what a rubric is. I know this is confusing. But the takeaway here is that these A.I. are incredibly literal. They will do exactly what you ask them to. So being as succinct yet descriptive as you can is imperative.
• You can modify results, which is what we will be doing in this example. When you get a result, you can then ask the A.I. to further modify that result in a new way. If the modification does not go the way you hoped, do not fear. All outputs are saved, and you can direct the A.I. back to previous iterations. Experimentation does not permanently alter your work.
• Do not be afraid to ask the A.I. for recommendations along the way! Depending on the assessment you are creating a rubric for it may have ideas for you to further sharpen the sort of criterion that you use.

Rubric Example – Science Presentation

I am as much a scientist as I am an instructional designer. For that reason I teach STEM related courses here at UNL. For this example, I am using a basic presentation (an assessment most of us are familiar with) as the foundation. I want to create a science presentation rubric that could be used in a 100-level course where non-STEM students may be taking the course for a gen-ed.

The first prompt I gave was Create a rubric with five criterion for a science presentation.

The result I got is exactly what I asked for. The A.I. picked five of the most common criteria and then described them. What you see that it did not do was place them in a standard rubric format of rows and columns. This is an example of the A.I. doing what I asked it in a very literal sense. It did not think or assume, it did as it was told.

This block of text is unwieldly and not the standard form. I then requested the A.I. modify the result into the tabular form as follows.

Can you put this in a table?

The resulting two-column table demonstrates that the A.I. was able to understand my request and separate the criteria from the descriptions. Now for some this rubric is all that is needed. A holistic rubric for your students to use to guide them in their process. Or at the very least this is a foundation for a holistic rubric that can be further augmented with new criteria and descriptions.

I wanted to take this rubric a step further to one that students see quite often. A rubric that sets point values to levels of each description. I asked it to this in the following prompt.

Can you make various grading levels in the descriptions?

Once again, the A.I. did exactly as I asked by describing five levels of grading from "excellent" to "poor". But it didn't put each point value and level description in its own column. Instead, it crammed all five levels for each criteria into the same table cell. But, this is easy to fix. I instruct it to do so.

Can you put the grading levels in their own columns?

What you see here is that the A.I. now placed each of the point values with description in their own columns. What you may see is missing is the column denoted only to the description of criteria. For the purposes of this demonstration, I did conduct the A.I. to bring that back as I found this generic and lower-level course rubric fine for my needs. But you may want to include that. You could create a new column, or could conduct the A.I. to place the description in the “criteria” space as is standard with Canvas and other rubrics. Your call.

What about getting this from ChatGPT to Canvas? That can be tricky. The first solution would be to copy and paste the content from one to the other. Which, while less time consuming than writing it all by hand, is still time consuming.

Another option still exists! You can request the rubric be crafted in CSV (comma-separated values) format which both Microsoft Excel and Canvas use.

I did find that sometimes the output might need to be augmented for it to perfectly transfer over. Further demonstrating that prior knowledge of a topic serves you in working with the A.I.

While this is a lower order rubric for a science presentation you can undoubtedly see the usefulness that A.I. can serve you in your course material. This rubric demonstrates how informed queries can be used to create a rubric for your discipline, assessment type, and level can be achieved. At the very least provide us with a foundation from where to start!

What else in your course could generative A.I. do to further best practices in education? Maybe we will have generative A.I. in the future that are trained specifically for that partnership. But that partnership and training will come from informed and experienced users. And we invite you to be those guiding conductors in A.I. in education.

Creation of Physics Symbology Resource

While working on a course, I found it difficult to find a resource that I assumed existed and would only be a quick Internet search away. Unfortunately, I found things that were similar to what I wanted, but not what I wanted. After about thirty minutes of work, I decided to see if ChatGPT could create it for me. What follows are the four prompts that I used in five minutes to create exactly what I needed.

The first prompt I used was, “Can you create [for] me a table of Greek letters, what their names are, and how they are used in physics?” The result got me close to what I wanted, a 3-column table with the column headings "Greek letter," "Name," and "Usage in Physics." However, notice that only the lower-case letter for the Greek symbol is used. I needed both upper as well as lower-case.

I then instructed the A.I., Can you create for me a table of upper an dlower case greek letters, what their names are, and how they are used in Physics?

It did not go quite as planned. I wanted the name column to remain the leftmost column and now the "Greek letter" and "Uppercase" column were duplicates. But, this shows how prompts can be misinterpreted and how you can further work with them in the AI interface to get them where you need them to be.

Can you move this name column of the above table to be the first column?

Before I tried to edit the columns I wanted to put them in the proper order. As you can see I request the name column to be moved from the second from the right, to the first column with the following prompt.

Finally, I asked the AI to remove the “upper-case” column and what I was left with was the resource that I had been hoping to find. I then used the disclaimer generated at the bottom of the first prompt and attached it to the final resource I created!