WEBVTT

1
00:00:00.690 --> 00:00:02.810
Nick Humensky: Alright. Thank you, Jorge.

2
00:00:03.250 --> 00:00:08.899
Nick Humensky: Good morning, everyone. My name is Nick Uminski, and I'm here with my colleague, Dr. Cindy Williamson.

3
00:00:09.080 --> 00:00:24.560
Nick Humensky: Tell you a little bit more about ourselves in just a moment. But 1st I want to thank everyone for being here today. We're very excited to showcase our collaborative dashboard effort, and hope that you walk away with some some practical insight and knowledge about how you could apply similar lessons to a project that you may be working on

4
00:00:30.590 --> 00:00:49.410
Nick Humensky: first.st Cindy is going to start out by telling you about how this project came to be, and then she's going to discuss how her team had a desire to move towards a more dynamic approach to analyzing evaluation data. And then I'm going to take over and I'll discuss the technical implementation and some of the details that I carried out to make this dashboard come to life.

5
00:00:49.650 --> 00:00:58.649
Nick Humensky: or these dashboards come to life, and then Cindy will end us up with a discussion about how this project helped her team align with campus and system level goals.

6
00:01:03.040 --> 00:01:13.969
Nick Humensky: So I am a data developer on the institutional effectiveness team under the greater academic affairs, research and student success team at the University of Tennessee System Administration.

7
00:01:14.430 --> 00:01:16.960
Nick Humensky: I handle a lot of dashboard creation

8
00:01:17.380 --> 00:01:23.469
Nick Humensky: back end data modeling automations ad hoc data requests things like that.

9
00:01:23.740 --> 00:01:24.560
Nick Humensky: Cindy.

10
00:01:25.530 --> 00:01:54.850
Cindy Williamson: So Jorge already mentioned it. But I'm the director of accreditation and assessment at Utc, and also our institutional accreditation liaison. So the data that we are talking about today is important for both of those things, accreditation and assessment. Our office is responsible for lots of data collection. So lots of surveys, including course, learning, evaluations, faculty rating of administrators, and then also responsible for the institutional

11
00:01:54.850 --> 00:02:08.779
Cindy Williamson: outcomes. Assessment process. That is, part of our saccoc requirements, but also is driven by the need to support students and and make sure that we're doing what we should

12
00:02:08.830 --> 00:02:09.940
Cindy Williamson: for them.

13
00:02:17.150 --> 00:02:20.100
Cindy Williamson: Okay? So this all started.

14
00:02:21.010 --> 00:02:28.049
Cindy Williamson: innocently enough. We were in. I think it was a power bi workgroup Zoom Meeting.

15
00:02:28.200 --> 00:02:43.450
Cindy Williamson: and I asked if we could get some help with power via because we had talked about in my office, wanting to be able to present data in a way that our stakeholders could interpret it, they could access it fairly easily.

16
00:02:43.560 --> 00:02:47.494
Cindy Williamson: so started with a conversation between

17
00:02:48.310 --> 00:03:01.290
Cindy Williamson: you know Jimmy and Nick's office. Rachel was there, and a couple of others, and then our office at Utc. So we really just described what we were thinking about, and

18
00:03:02.160 --> 00:03:11.939
Cindy Williamson: then also talked as as groups back and forth about you know, questions that came up, or ideas that we had along the way.

19
00:03:12.130 --> 00:03:38.519
Cindy Williamson: and we all know how important communication is. And when we were planning this presentation, I I it just made me think about. Yes, that's how this all started was clear communication, answering questions. Making sure that everybody understood the need. And then, on our end, making sure that we understood what was possible and how how we could make these dashboards look you know, based on

20
00:03:39.110 --> 00:03:41.660
Cindy Williamson: power bi parameters.

21
00:03:41.930 --> 00:03:49.839
Cindy Williamson: So we're able to you know, communicate openly and talk about the project. But then

22
00:03:50.660 --> 00:04:12.580
Cindy Williamson: there's so much data, all the data. So it was interesting to to kind of step back and think about our data from a different perspective. We decided to start with our outcomes, assessment information course, learning evaluations and senior exit exams, and we'll we'll go into a little more detail on those here. In just a second

23
00:04:16.720 --> 00:04:22.010
Cindy Williamson: already mentioned communication, we had an excellent cycle of communication back and forth.

24
00:04:23.790 --> 00:04:26.360
Cindy Williamson: We answered questions, provided data.

25
00:04:27.240 --> 00:04:38.740
Cindy Williamson: And I've mentioned this already. But we thought about what we were doing as part of our data collection process and presenting that data from a place that we don't often

26
00:04:38.890 --> 00:04:45.323
Cindy Williamson: get to be. You know, a lot of times we're distracted by the process. How are we gonna make this happen.

27
00:04:45.920 --> 00:04:55.491
Cindy Williamson: and here's what we want. Let's get it. But really taking a step back and thinking about possibilities, and the best way to make this

28
00:04:56.210 --> 00:05:11.669
Cindy Williamson: it well, make it an efficient process, but also something that our stakeholders could use. When they needed to. So really, thinking about the the bigger picture in order to make data informed decisions across campus.

29
00:05:19.100 --> 00:05:25.989
Cindy Williamson: So the 3 dashboards that we started with were the Ets proficiency profile, which is our senior exit exam.

30
00:05:26.110 --> 00:05:30.689
Cindy Williamson: It's required that all graduating seniors take this exam.

31
00:05:31.240 --> 00:05:38.949
Cindy Williamson: We do 3 administrations of it. Well, 3 semesters outcomes assessment information.

32
00:05:39.562 --> 00:05:41.070
Cindy Williamson: That's a lot.

33
00:05:41.470 --> 00:05:49.400
Cindy Williamson: Our office. We have a team. Well, we have 5 people and 3 who work specifically on outcomes, assessment review and feedback.

34
00:05:49.966 --> 00:06:07.820
Cindy Williamson: We report, we report these data multiple times a year, and then, of course, also report to Saccoc as required. And I left out that our our senior exit exam is also reported to Tec. So not just something that we use at our institution, but as part of our quality assurance, funding.

35
00:06:08.682 --> 00:06:19.739
Cindy Williamson: course, learning evaluations. Yes, results are available to faculty department heads, deans, administrators, other stakeholders. But we wanted a way

36
00:06:20.269 --> 00:06:41.940
Cindy Williamson: for these stakeholders to easily access what they need. What we have not right now is fine. Our platform is just fine. But the reporting capability is limited, and so we thought that a dashboard would be a great way to present that information in a way that can be interpreted by those who need who need to use the data.

37
00:06:45.940 --> 00:06:49.984
Cindy Williamson: Okay, this is where my sob, story starts.

38
00:06:50.830 --> 00:06:58.460
Cindy Williamson: so senior exit exam. Results were previously presented in excel. We'll show you some examples in a minute. But

39
00:06:59.070 --> 00:07:01.983
Cindy Williamson: yeah, just don't cry. Okay. They're bad.

40
00:07:03.130 --> 00:07:09.050
Cindy Williamson: and the way it's presented, which you'll see, it's not easy to interpret results. And

41
00:07:09.290 --> 00:07:29.549
Cindy Williamson: we want this data to be used, not just you know, by OA for T Hec purposes. But this is a lot of good information that we can use programmatically, departmentally. You know, in order to make changes, make improvements, data informed decisions.

42
00:07:31.100 --> 00:07:34.506
Cindy Williamson: so yeah, we really wanted to make sure that those who needed

43
00:07:35.700 --> 00:07:41.950
Cindy Williamson: that maybe even didn't know they needed it. But those who we thought should be using the data would have access to it.

44
00:07:46.560 --> 00:07:49.736
Cindy Williamson: This is it. I'm sorry. It's graphic.

45
00:07:50.320 --> 00:08:00.990
Cindy Williamson: it's it's terrible. This is what has been on our our website for several years. We usually have at least 5 years of data up

46
00:08:01.200 --> 00:08:19.719
Cindy Williamson: at a time. So imagine as a department head or faculty member or dean, you know, trying to make sense of this information. It's there we have proficiency scores. We have, you know, mean test scores mean scores for each subject on the senior exit. Exam. But

47
00:08:20.502 --> 00:08:34.850
Cindy Williamson: it's it's it's not fun. And I know that there have been difficulties in people accessing the results and getting what they need. But just like with these other data sets, we were constantly

48
00:08:34.909 --> 00:08:50.399
Cindy Williamson: and probably more for course, learning evaluations than anything else, but providing data through, you know, from ad hoc requests. So we had our process. But then, on top of that, we we have multiple requests for the data in a different format.

49
00:08:55.792 --> 00:08:57.840
Cindy Williamson: Our outcomes assessment data.

50
00:08:58.060 --> 00:09:07.290
Cindy Williamson: This is a relatively new you know, data set that we had. We have worked for about 10 years.

51
00:09:07.804 --> 00:09:14.015
Cindy Williamson: To get to where we are now with outcomes assessment. So we've improved that process.

52
00:09:14.720 --> 00:09:38.510
Cindy Williamson: I mentioned that there are 3 of us who work with the outcomes, assessment information and really 2 fantastic outcomes, assessment management analysts who review every piece of outcomes assessment information for every program, department, college office, everybody across campus, academic student support, administrative.

53
00:09:39.061 --> 00:09:57.150
Cindy Williamson: So we're talking about around 230 separate areas, separate offices or departments, and we have about 220 outcomes assessment contacts that we send information to send data to send our assessment results to our feedback.

54
00:09:57.833 --> 00:10:04.556
Cindy Williamson: So we do provide specific, detailed feedback. We'll see an example of that here in just a second. But

55
00:10:05.920 --> 00:10:23.470
Cindy Williamson: we want to know where and how assessment occurs. Yes, we want these areas to highlight strengths and weaknesses in order to see where they can focus their efforts and where more resources may be needed. There may be a need for changes to the curriculum

56
00:10:23.890 --> 00:10:30.574
Cindy Williamson: changes to their particular assessment process. All kinds of good information that can be gained

57
00:10:31.130 --> 00:10:33.240
Cindy Williamson: from this outcomes assessment data.

58
00:10:36.850 --> 00:10:55.493
Cindy Williamson: So this isn't as bad as the senior exit data, but the the 1st screenshot you see at the top, the top on the right. That is our internal spreadsheet. So we had a spreadsheet, have still have a spreadsheet that lists all departments, all programs

59
00:10:56.070 --> 00:11:12.630
Cindy Williamson: and then contacts for those departments, and then across the top you'll see that the columns are. It's kind of hard to see on here, I'm sure, but the columns include deadlines, dates where certain pieces of our outcomes assessment information is

60
00:11:13.035 --> 00:11:22.769
Cindy Williamson: so when it's supposed to be reported, so we make notes on here, follow up with those who don't have what what they're supposed to have at the time.

61
00:11:23.261 --> 00:11:31.700
Cindy Williamson: And then that second screenshot is our outcomes assessment rubric. So that's 1 of the big improvements that we've made.

62
00:11:31.730 --> 00:11:54.509
Cindy Williamson: and that's been in the past 3 or 4 years. It's a rubric that has dimensions aligned with each requirement for the outcomes assessment process. So each outcome that's developed is supposed to have specific information. And so that's what we're looking at with that rubric.

63
00:11:55.084 --> 00:12:05.265
Cindy Williamson: Nick, you can go on to the next one. So we have that rubric feedback that we provide. But then also we provide

64
00:12:05.900 --> 00:12:29.149
Cindy Williamson: more qualitative feedback. So specifically like, here's your score. But here's specifically what we were looking for and what we didn't see. Or here's where you did well keep doing that. So we send all this feedback via email. We also post it in anthology planning, which is our platform that we use for outcomes assessment data collection.

65
00:12:30.630 --> 00:12:47.489
Cindy Williamson: and this information is, yes, helpful to individual departments, but we were thinking that having a dashboard would make it more accessible for the larger units. So the the colleges and units

66
00:12:48.150 --> 00:12:59.519
Cindy Williamson: they could access the information. And then, like I already said, you know, make data, informed decisions, improvements. And those responsible for outcomes assessment

67
00:12:59.760 --> 00:13:14.219
Cindy Williamson: typically only see their information, their feedback, that we provide. So this is a way to for multiple departments to have some kind of comparison as as far as what they're saying for themselves.

68
00:13:18.430 --> 00:13:24.048
Cindy Williamson: And everybody's favorite data collection process is course, learning evaluations.

69
00:13:24.960 --> 00:13:46.389
Cindy Williamson: it's always a hot topic, I'm pretty sure, on every campus across the country, probably across the world anyway. So currently, administrators, deans, department heads faculty can access results through the platform. I mentioned, though, that it is not user friendly and

70
00:13:46.784 --> 00:13:56.639
Cindy Williamson: the reports that can be generated through that platform are are lacking. So we have a ton of ad hoc requests. That come through

71
00:13:57.380 --> 00:14:01.904
Cindy Williamson: for this cle data in in different formats.

72
00:14:03.650 --> 00:14:24.239
Cindy Williamson: and there is comparative data in that platform. But that's 1 of the main things that I think we're asked about even though they can access it. It's not easy to access and having something right there already available, ready for people to use without making a separate request. I mean, that just just sounds like.

73
00:14:24.450 --> 00:14:32.670
Cindy Williamson: you know, the ultimate way to be with your your data. Processes. But so

74
00:14:33.260 --> 00:14:49.960
Cindy Williamson: lots of reasons that cle data is important. Jorge actually hit on that. And in his introduction, this is this is for faculty. They can use it in different ways, including, you know, during their promotion and tenure efforts.

75
00:14:52.890 --> 00:15:06.760
Cindy Williamson: These next couple of slides are examples again. Still, not as bad as senior exit exams. But it's not. It's not great. We you can see that there's so many different ways. We're asked to format the results

76
00:15:07.080 --> 00:15:11.190
Cindy Williamson: on the left hand side. You'll see that it's by

77
00:15:11.780 --> 00:15:15.090
Cindy Williamson: cle question, and then also about term

78
00:15:16.094 --> 00:15:24.159
Cindy Williamson: and then on the right hand side, we've also got it broken down by response type and question.

79
00:15:24.350 --> 00:15:34.840
Cindy Williamson: Then on the bottom, it's, you see, the comparison data there. So we have the program, the department, the college, and then the university. But that's also broken down by semester

80
00:15:36.360 --> 00:15:38.769
Cindy Williamson: and then this next one

81
00:15:40.330 --> 00:15:48.125
Cindy Williamson: is is broken down. Even further, we have as part of this request. Well, both of these requests. I think we

82
00:15:48.780 --> 00:16:02.690
Cindy Williamson: provide individual instructor data. And and that's for the department, right? That's not something that that's going out to everybody. But we have the app, the mean scores by course, modality also by term.

83
00:16:03.725 --> 00:16:08.440
Cindy Williamson: And then we've got the response

84
00:16:08.890 --> 00:16:23.899
Cindy Williamson: averages means by term and by question. So anyway, you get the picture like this is just a lot of data and having it displayed in a way that is functional, that can be interpreted and used is is what we needed.

85
00:16:27.900 --> 00:16:31.359
Cindy Williamson: And I told Nick that this is probably an

86
00:16:32.010 --> 00:17:00.978
Cindy Williamson: an underestimate of how much time this is actually saving us. When you think about updating spreadsheets with new data multiple times a year. Senior exit, we only do once a year. But then outcomes assessment. There's 3 or 4 updates a year that we do to that data. And it's aligned with those deadlines that that they have throughout the cycle. And then, Cle, data is is updated all the time as well.

87
00:17:02.020 --> 00:17:07.709
Cindy Williamson: at the least 3 times a year, once for each term.

88
00:17:13.190 --> 00:17:13.889
Nick Humensky: Alrighty.

89
00:17:14.770 --> 00:17:16.250
Nick Humensky: Thank you, Cindy, for that.

90
00:17:16.440 --> 00:17:23.990
Nick Humensky: Now I'm going to take over, and I'm going to walk you through the pieces of the technical implementation, and how we got this dashboard to be

91
00:17:27.420 --> 00:17:38.199
Nick Humensky: alright with any project like this or any sort of dashboard project. It's always good. It's a good idea to start out with a solid requirements gathering meeting.

92
00:17:38.350 --> 00:17:43.509
Nick Humensky: So we have 3 dashboards, we know. And for each I want detail on the following items.

93
00:17:44.560 --> 00:17:59.090
Nick Humensky: data source. So where is this coming from? Is it from a software? Is it a flat file or some sort of Api that we can connect to? Also does the structure of the data change over time. For example, do the number of columns or fields shrink or grow?

94
00:17:59.590 --> 00:18:06.030
Nick Humensky: What is the frequency in which you're going to be receiving the data. Is it monthly, yearly or by term?

95
00:18:06.150 --> 00:18:17.509
Nick Humensky: This helps me decide whether or not I want to have the data refresh pretty much instantly if it's very frequent, or have it set up on some sort of schedule to refresh the data more periodically, if it's less infrequent.

96
00:18:19.200 --> 00:18:29.319
Nick Humensky: what is the volume of the data. So each time that you receive a batch in the cadence of the frequency, how many rows are coming in with it? Is it hundreds, thousands, millions.

97
00:18:29.960 --> 00:18:33.569
Nick Humensky: Spoiler, alert, we went with storing the data on

98
00:18:34.280 --> 00:18:40.040
Nick Humensky: sharepoint and excel files. And what this volume of data piece helped me decide was.

99
00:18:40.500 --> 00:18:55.359
Nick Humensky: if I wanted to keep the data all in one spreadsheet and just append rows to it. Or if there's just too much too many rows and a lot of transformations that needed to happen to where I just wanted the stakeholder to go in and be able to drop a file and have power bi refresh it.

100
00:18:55.700 --> 00:19:20.809
Nick Humensky: And to that matter, the transformations I like to get with the stakeholder and understand exactly the process that they have where they take the data from the raw format to the presentable format. I like to mimic their actions as closely as possible in whatever automation tool I'm using in this case, it's power query. It's similar to what Justin said yesterday. If you were on for the panel discussion, our job is to help the stakeholder

101
00:19:21.050 --> 00:19:30.600
Nick Humensky: not worry about the data or any of the manual pieces. Let us take care of all of the automation so that you can focus on what your area of expertise is.

102
00:19:31.040 --> 00:19:33.480
Nick Humensky: And then finally, security.

103
00:19:34.640 --> 00:19:38.839
Nick Humensky: Are there any sensitive pieces of information that we want to hide or protect?

104
00:19:38.980 --> 00:19:42.959
Nick Humensky: Or is there any sort of row level security that we want to implement, such as

105
00:19:43.300 --> 00:19:46.389
Nick Humensky: keeping departments from seeing each other's data.

106
00:19:46.850 --> 00:19:49.080
Nick Humensky: And so with that.

107
00:19:50.240 --> 00:20:03.119
Nick Humensky: this is sort of a high level piece or screenshot of what I have for all 3 of the dashboards the course learning, evaluation, and the senior exit exam. Dashboards both come from an external software.

108
00:20:03.220 --> 00:20:15.630
Nick Humensky: Unfortunately, there was no way to directly connect to them and get automatic access to the data. So we had to go with the next thing. The next best thing, which is a export to a tabular data source such as a spreadsheet.

109
00:20:16.010 --> 00:20:20.859
Nick Humensky: The outcomes assessment dashboard is just a manually created spreadsheet maintained by Cindy's team

110
00:20:21.300 --> 00:20:26.969
Nick Humensky: for the frequency of the data. Both the Senior exit Exam and outcomes assessment dashboards

111
00:20:27.810 --> 00:20:37.520
Nick Humensky: are updated about once a year. The outcomes assessment might be Updated 2 or more. But the course. Learning evaluation is a different story that is updated once per term.

112
00:20:38.040 --> 00:21:05.059
Nick Humensky: Again, for the volume outcomes assessment and senior exit exam are similar. It's just hundreds high, hundreds of rows. And then course, learning evaluation is a massive data set. It's got tens of thousands of rows in each file that's added to the sharepoint folder, as far as transformations go, outcomes. Assessment and senior exit exam are pretty simplistic. You're just adding new rows to the existing spreadsheet. But the course, learning evaluation requires some complex transformations, and we'll talk about that in a moment

113
00:21:05.150 --> 00:21:15.390
Nick Humensky: and for security. But the course learning, evaluation and the senior exit exam dashboards have some names in them. The the free text questions might have

114
00:21:15.670 --> 00:21:27.669
Nick Humensky: the professor names in the course, learning evaluation, and the senior exit exam records the students, 1st and last names in in separate fields whenever they take the exam, and we take measures to protect that which I'll showcase in a second

115
00:21:28.750 --> 00:21:39.210
Nick Humensky: with all of that in place. I determined that the easiest and best way to move forward with this would be to store the data in excel files in a structured manner in a sharepoint folder.

116
00:21:39.420 --> 00:21:40.270
Nick Humensky: So

117
00:21:40.860 --> 00:22:03.439
Nick Humensky: you have the General Document library that comes with standard sharepoint sites underneath that I created a folder called assessment data, dashboard data sources. And I created this to house everything under there and separate it from any other projects that they might have within their team's sharepoint site, but I wanted them to be able to have access to the site so that they could go in and drop the data files in there

118
00:22:03.950 --> 00:22:10.460
Nick Humensky: underneath that I have a folder for each of the 3 dashboards, and then you'll notice

119
00:22:11.970 --> 00:22:17.599
Nick Humensky: there are right underneath the level, right underneath the dashboard folder are data files

120
00:22:18.690 --> 00:22:24.099
Nick Humensky: for each of the dashboards, and then I also created a separate dimensional folder

121
00:22:24.250 --> 00:22:30.460
Nick Humensky: that will allow you to drop in any sort of if you're familiar with standard data modeling or reporting

122
00:22:31.341 --> 00:22:35.899
Nick Humensky: acumen dimensional tables just provide additional context

123
00:22:36.380 --> 00:22:39.109
Nick Humensky: and labels for the factual data.

124
00:22:43.790 --> 00:22:54.460
Nick Humensky: So now that I have the data stored, it's time to start pulling it into power bi. And whenever you create a connection to data and power bi, you want to make sure that you're setting up authentication and privacy levels appropriately.

125
00:22:55.500 --> 00:23:04.989
Nick Humensky: Here's a reiteration of the security. You have the names in both the course, learning, evaluation and the senior Exit Exam. Nothing really an outcomes assessment. It's just department names and scores

126
00:23:05.486 --> 00:23:17.640
Nick Humensky: for the authentication on all 3. I wanted to do Microsoft credentials. It makes the most sense because everyone who's going to be using this as part of the University of Tennessee system, so they will be on the Microsoft tenant

127
00:23:18.120 --> 00:23:19.550
Nick Humensky: for privacy levels.

128
00:23:20.210 --> 00:23:21.949
Nick Humensky: I went with

129
00:23:22.290 --> 00:23:28.769
Nick Humensky: private, organizational and private. There's 4 different privacy levels. Whenever you're connecting data in power. Bi, there's none.

130
00:23:29.000 --> 00:23:31.449
Nick Humensky: public, organizational and private.

131
00:23:31.700 --> 00:23:41.459
Nick Humensky: The reason I went with private for course, learning, evaluation and senior exit exam is because of the names. And what private means is that it just isolates that data. Query, that specific data query

132
00:23:41.740 --> 00:23:49.360
Nick Humensky: from being merged or appended to any other data sets and creating some sort of unwanted inference about maybe the student

133
00:23:49.950 --> 00:23:56.839
Nick Humensky: so the private isolates it from being connected or merged with an even other private data sets

134
00:23:57.600 --> 00:24:09.240
Nick Humensky: for outcomes. Assessment. I went with organizational. That's a step up from private. It's just for more trusted internal resources. I didn't. Blanket set everything to private, because I wanted to be intentional about my decisions of

135
00:24:09.370 --> 00:24:12.159
Nick Humensky: the level of privacy that I set for the dashboards.

136
00:24:16.680 --> 00:24:21.460
Nick Humensky: So with the dashboard or with the senior exit exam and outcomes assessment.

137
00:24:21.890 --> 00:24:32.270
Nick Humensky: it's pretty simple. You just connect to a single spreadsheet, and then there's minimal transformations that need to be done to it. But the course learning evaluation is a monster, and I'm going to show you

138
00:24:32.730 --> 00:24:44.639
Nick Humensky: some of the steps that I took to be able to create make this digestible format for power. Bi. Now, here we have a it's a kind of a cumbersome looking spreadsheet, but I'll I'll describe it the best I can.

139
00:24:44.840 --> 00:25:02.030
Nick Humensky: This is just one terms, file that comes from Cindy and her team. For one term. Of course, learning evaluation survey data. Within each file there are different sections or templates of data responses. And that's just for the different colleges that

140
00:25:02.130 --> 00:25:10.339
Nick Humensky: they might have different templates, which just means they have some additional questions that they ask for the students within that particular particular program.

141
00:25:10.900 --> 00:25:18.990
Nick Humensky: They all start out like this where there's it skips a few rows and

142
00:25:20.160 --> 00:25:29.590
Nick Humensky: it has a little header here. It says raw results for template brsc. That's just the honors college at Utc. It'll skip a row, and then it has the

143
00:25:30.520 --> 00:25:35.950
Nick Humensky: headers down here like this, and then so all the responses

144
00:25:36.150 --> 00:25:43.090
Nick Humensky: and brsc usually has a couple of 100 rows. I did this little row skip, to try and show

145
00:25:43.220 --> 00:25:48.219
Nick Humensky: how like a different section in the data without having it scroll out and being extremely scrunched.

146
00:25:48.670 --> 00:25:51.539
Nick Humensky: So this right? Here is the

147
00:25:51.940 --> 00:25:54.760
Nick Humensky: last row of the brsc data.

148
00:25:55.470 --> 00:26:01.589
Nick Humensky: and then the next section starts raw. Result, raw results template for course, evaluation skips row

149
00:26:02.240 --> 00:26:08.580
Nick Humensky: more more data, and there's a varying number of sections within each spreadsheet. There is a varying number of rows.

150
00:26:08.720 --> 00:26:15.490
Nick Humensky: and there's a varying number of columns like I mentioned, because some might have additional columns or additional questions.

151
00:26:15.800 --> 00:26:17.199
Nick Humensky: So if you notice

152
00:26:19.090 --> 00:26:25.880
Nick Humensky: starting at this fall, 2020 right here and all the way to the right are all question columns, and it could go to column

153
00:26:26.580 --> 00:26:29.289
Nick Humensky: Z. It could go to column A, B,

154
00:26:29.450 --> 00:26:37.350
Nick Humensky: and as such, when you pull this into power. Bi, you can't simply filter out the blanks and just stack them on top of each other. We have to do a little transformation.

155
00:26:37.520 --> 00:26:45.389
Nick Humensky: Do some unpivoting and get an equal number of fields so that you can create one long, narrow table that's more easily digestible by power. Bi

156
00:26:46.130 --> 00:26:48.260
Nick Humensky: and I included this little graphic

157
00:26:49.570 --> 00:26:58.820
Nick Humensky: to help you visualize what unpivoting is. If you don't know what it means. I'm sure everyone knows what it means, but I wanted to include it just in case someone might not be familiar with unpivoting.

158
00:26:59.030 --> 00:27:01.470
Nick Humensky: Let's say you have a

159
00:27:02.670 --> 00:27:09.520
Nick Humensky: survey that's sent out. And these 3 people respond to the 3 questions. Here you have these descriptive fields, the dimensional fields.

160
00:27:09.900 --> 00:27:19.799
Nick Humensky: such as their name and department? And then you have all the questions that they answered the question. Headers are the attributes, and then the respective values that they answer for each question below

161
00:27:19.940 --> 00:27:26.019
Nick Humensky: unpivoting. What that does is. It transposes the column headers down to a single column like this.

162
00:27:26.370 --> 00:27:28.220
Nick Humensky: And then the matching values.

163
00:27:28.740 --> 00:27:39.979
Nick Humensky: This does create create some duplication and redundancy in the descriptor fields. But it's necessary for this particular data set. And it's a much easier format to pull into power bi to work with.

164
00:27:41.790 --> 00:27:51.000
Nick Humensky: So this is what I this is what I had in. This is just one of the sections that I would work on in the Cle. Dashboard. So I keep all the

165
00:27:51.560 --> 00:28:04.009
Nick Humensky: columns between term and professor name, and then anything to the right of that is going to be an attribute column, and I have to use some complex power query to be able to dynamically figure out where the end is, and then unpivot it.

166
00:28:06.080 --> 00:28:13.669
Nick Humensky: And so here's what all the power query steps for that look like. I know this is a lot, and I don't expect you to read, and really

167
00:28:14.420 --> 00:28:23.730
Nick Humensky: I don't expect you to read all of it. But I have this in here for just a little heads up for best practices in terms of naming your steps when you're working in power. Query.

168
00:28:24.010 --> 00:28:44.079
Nick Humensky: You want to be as descriptive and specific as possible to help yourself whenever you come back in several months at a time to remember exactly what's happening in the power query. And also if someone else inherits the dashboard, they'll be able to go in and troubleshoot a step, because they'll be able to see what's happening in the steps.

169
00:28:45.310 --> 00:28:46.570
Nick Humensky: I left

170
00:28:47.750 --> 00:29:10.790
Nick Humensky: these 1st few steps as their general, the generic names that are generated by power, bi or power query. For a couple of reasons. One. These are automatically generated whenever you connect a sharepoint file to power Bi. They don't really do anything for the purposes of troubleshooting. It's just creating some additional helper files to pull the data in. And

171
00:29:10.860 --> 00:29:21.577
Nick Humensky: also I wanted to showcase how these generic step generic name steps don't really mean anything so filtered hidden files. One? What does that mean? Well,

172
00:29:22.280 --> 00:29:30.569
Nick Humensky: yeah, you want to be specific, because there could be multiple filtered rows, filtered rows, one filter rows, 2 things like that. And so everything below that

173
00:29:32.070 --> 00:29:44.229
Nick Humensky: and over to the right are all the steps that I created for this particular data set in order to unpivot all the columns, all the sections, and combine all the files together.

174
00:29:47.380 --> 00:29:53.400
Nick Humensky: And so, after all is said and done, you have one huge table that has.

175
00:29:53.880 --> 00:29:58.569
Nick Humensky: I think, upwards of 3.9 million of rows or 3.9 million rows.

176
00:29:58.980 --> 00:30:02.829
Nick Humensky: That's quite a lot of data to work with. And because the

177
00:30:03.730 --> 00:30:09.289
Nick Humensky: the value column is going to have a combination of both. The free text answers as well as the numeric answers.

178
00:30:09.530 --> 00:30:18.100
Nick Humensky: it's quite difficult to perform mathematical Dax calculations on fields like that, unless you want the measures to be extremely cumbersome

179
00:30:18.470 --> 00:30:40.990
Nick Humensky: as such. I separated the big table that's aggregated into 3 fact tables, and I did this to make the Dax easier, but group them logically together. So it's a little easier and more clear to work with. So I have a table that has groups together all of the likert questions. And that's all of those questions where you rank between strongly disagree or strongly agree.

180
00:30:41.160 --> 00:30:46.880
Nick Humensky: We have fact text, and that's just all the free text responses and questions. And then we have.

181
00:30:47.250 --> 00:30:54.020
Nick Humensky: I just call this counts. It's any other remaining question. They're pretty much multiple choice questions, such as.

182
00:30:54.140 --> 00:30:58.950
Nick Humensky: what modalities did you use to be successful in this course? Choose all that apply items like that.

183
00:30:59.120 --> 00:31:09.909
Nick Humensky: And then this 4th one over here is a separate data sort set that comes from the same tool as the surveys. But it's just a different spreadsheet. It's for the response rates, and I have that separately. But I

184
00:31:10.130 --> 00:31:17.979
Nick Humensky: am able to connect them all to the dimensional tables based on term and college and department connections like that.

185
00:31:22.190 --> 00:31:27.130
Nick Humensky: So once I have the data in power Bi, and you start creating measures. And you have all that good stuff

186
00:31:27.420 --> 00:31:43.839
Nick Humensky: you want to start worrying about the visual styling. And so I got some branding assets from Cindy, including the Utc approved colors and their hex codes. And then I also asked for some vector images, Logos of her office, so that I can include that on the dashboard.

187
00:31:43.840 --> 00:32:01.389
Cindy Williamson: Hey, Nick? I just wanna add in really quick this is kind of funny. So when Nick asked for the graphics or Logos, I asked him which ones he wanted, and he told me, but I had no idea what he was saying. Like the words did not, so I was like, well, which ones are those, and he's like, just send them all. Just send them all to me.

188
00:32:01.580 --> 00:32:02.170
Cindy Williamson: So.

189
00:32:03.040 --> 00:32:07.699
Nick Humensky: Yeah, they these were helpful to add that sort of personal touch to the dashboards.

190
00:32:09.730 --> 00:32:31.890
Nick Humensky: So here is an initial iteration of the outcomes assessment dashboard. It's called. I initially had it called the Assessment Review. But it's now the assessment or the outcomes assessment dashboard where I had across the top the all the different filters for the dashboard. So that way I could create as much real estate in the body area, keep it wide, and have all the visuals large, so that people can see them.

191
00:32:32.510 --> 00:32:37.339
Nick Humensky: But I did the same thing for the senior exit exam. Where I had the filters across the top

192
00:32:37.640 --> 00:32:46.609
Nick Humensky: for the course, learning evaluation. There were too many filters to have it horizontally across the top, so I had to make them vertical, and I didn't want to have 2 dashboards where

193
00:32:46.840 --> 00:32:51.629
Nick Humensky: they were the filters were horizontal, and then one was vertical. I wanted everything to be consistent.

194
00:32:51.930 --> 00:32:56.620
Nick Humensky: so I took some time to think it through. And this is the format that I ended up coming up with

195
00:32:57.460 --> 00:32:58.810
Nick Humensky: is the following.

196
00:32:59.160 --> 00:33:17.589
Nick Humensky: I have a narrower navigation bar on the left where I'll have some buttons, such as a filter button to show and hide a filter pane so that you can apply filters. But you don't have to have a consistent, persistent filter pane on the left that takes up space where you could have more visuals.

197
00:33:17.710 --> 00:33:24.970
Nick Humensky: and then other page navigation icons, and then a button for a tutorial that'll show you how to walk through the dashboard and use it

198
00:33:25.200 --> 00:33:35.429
Nick Humensky: up on the top. We have a header section. That's where I have the dashboard title, a label on when the data was last updated, and then I also will have a text box that contains

199
00:33:35.670 --> 00:33:46.369
Nick Humensky: which filters have been applied, so that after you hide the filter pane you can understand and remember what the context is that you're looking at. So you don't have to keep toggling the pane back on, back and forth.

200
00:33:46.770 --> 00:33:48.880
Nick Humensky: And then just a main data section.

201
00:33:49.770 --> 00:34:18.579
Nick Humensky: Also fun fact. I found out that you can create backgrounds and pretty looking visuals in Powerpoint. This background was actually created in Powerpoint. You can do a little more complex, and fancy visuals, such as the gradient borders and transparency effects within Powerpoint, and then save the slide as an Svg. And set that to your canvas background and do an Svg. Just so it doesn't lose the quality whenever it scales and expands to a bigger size.

202
00:34:18.870 --> 00:34:41.380
Cindy Williamson: Hey, Nick, one other quick thing. So it was interesting, as Nick was doing all this stuff. When he would find something new or create something. He he would, you know, message me and be like, can you get on real quick? I got something to show you. I just found this out or so it was like, I don't know. It's just kind of refreshing to to know that like this was exciting right on both

203
00:34:41.389 --> 00:34:46.920
Cindy Williamson: things. This wasn't just like me being excited about what what the final result was. Gonna be so.

204
00:34:47.510 --> 00:34:48.679
Nick Humensky: Definitely.

205
00:34:49.370 --> 00:34:57.470
Nick Humensky: And so just to reiterate, here's that initial iteration of the outcomes assessment dashboard the before.

206
00:34:57.570 --> 00:35:01.274
Nick Humensky: And then here's the after. It's a little more clean looking with the

207
00:35:02.430 --> 00:35:06.390
Nick Humensky: gradients from the approved Utc colors, and

208
00:35:08.070 --> 00:35:13.970
Nick Humensky: Yeah, yeah, right? It's it's super fancy. So over here. This is the

209
00:35:14.430 --> 00:35:39.830
Nick Humensky: label that shows when it was last updated. Here is the text box that shows the filters that are applied, and this is the filter button that shows and hides the filter pane. Another little thing that, I added, is this number here to just add another visual cue to remind you how many dimensions have been filtered. The outcomes assessment doesn't have any other pages. It's just one. So there's nothing here, but there is the how to use button down here. I'll show you what that looks like in a moment.

210
00:35:42.120 --> 00:35:46.819
Nick Humensky: Here's the senior exit exams data, same format, similar format.

211
00:35:47.440 --> 00:35:58.030
Nick Humensky: this one actually has multiple pages. So right now, you're viewing the scores. But there's also a little radar gauge, but button to go to the proficiency profile.

212
00:36:00.490 --> 00:36:06.290
Nick Humensky: and finally the the big one, the course learning evaluation.

213
00:36:06.580 --> 00:36:09.760
Nick Humensky: This has the most pages, and

214
00:36:10.400 --> 00:36:17.090
Nick Humensky: on the front home page. I wanted to include the the the minimum required

215
00:36:17.380 --> 00:36:25.610
Nick Humensky: measures that Cindy asked for. Just so it's right front and center for people to be able to go in and look at, and slice and dice. However, they see fit with the filters

216
00:36:26.600 --> 00:36:33.130
Nick Humensky: whenever you click on the filter button. This is what it looks like pops up, and you can adjust the slicers.

217
00:36:33.450 --> 00:36:38.810
Nick Humensky: You can see that they are applied up in the text box behind a little transparent background, just to

218
00:36:39.160 --> 00:36:43.530
Nick Humensky: show that something is currently in the front foreground of the visuals.

219
00:36:43.890 --> 00:36:45.300
Nick Humensky: and then you can hide it.

220
00:36:45.850 --> 00:36:50.799
Nick Humensky: And then this is what the tutorial looks like when you click on this, how to use button down here.

221
00:36:51.000 --> 00:37:13.350
Nick Humensky: It brings up this page that says, you know, welcome to the dashboard. I want a tour, or you can just click no and go right back into working. But whenever you click this, the following happens, it starts a little sequential coach mark one of those sort of self guided tutorials on how to use the dashboard. I didn't include a screenshot for every one of them, but I have it walked through

222
00:37:14.680 --> 00:37:16.890
Nick Humensky: a few pieces for the navigation bar.

223
00:37:17.390 --> 00:37:24.150
Nick Humensky: and then it'll walk through the filters. It'll walk you through how to use the filter page. And then the

224
00:37:24.860 --> 00:37:28.679
Nick Humensky: this little section is for the data, and it just walks you through looking at the data.

225
00:37:30.800 --> 00:37:33.530
Nick Humensky: So we're really excited to get these out and rolling.

226
00:37:34.810 --> 00:37:39.810
Nick Humensky: and I'm going to pass it back to Cindy to discuss the final strategic alignment steps.

227
00:37:40.850 --> 00:37:43.155
Cindy Williamson: Wanted to mention to Nick that

228
00:37:43.800 --> 00:37:51.312
Cindy Williamson: That help feature in the dashboards. Dashboards, I think, is something that's going to be really helpful.

229
00:37:51.900 --> 00:38:01.519
Cindy Williamson: of course we're there. Nick's there. Whoever is there to answer questions. But I think, having kind of a tutorial like Walkthrough situation is.

230
00:38:01.820 --> 00:38:07.045
Cindy Williamson: it's gonna be really nice. So yeah, strategic alignment.

231
00:38:08.380 --> 00:38:11.529
Cindy Williamson: When we 1st started this, I obviously did not.

232
00:38:12.690 --> 00:38:15.489
Cindy Williamson: Excuse me, I obviously did not think about it on

233
00:38:15.720 --> 00:38:28.096
Cindy Williamson: on this level, right about this collaboration between campus and system. But I am glad to have had the opportunity to think about it in that way, because this is something that's really important.

234
00:38:28.830 --> 00:38:41.510
Cindy Williamson: of course, ideally, we want goal alignment, right between the system utc our office, any office on our campus but that cross campus collaboration, I think, is super important.

235
00:38:42.020 --> 00:39:08.139
Cindy Williamson: So open data sharing across departments and units and then also system wide. Yes, we report data to system. Yes, we report to T. Hec, but I think that reciprocal data transfer. And while in this case, like what Nick was doing wasn't technically, you know, data transferred. It was a back and forth process. That

236
00:39:08.560 --> 00:39:17.265
Cindy Williamson: required efficient communication in order for us to you know, both get what we need for our offices for our

237
00:39:17.890 --> 00:39:21.237
Cindy Williamson: you know, employers to get what we need. So

238
00:39:21.900 --> 00:39:43.040
Cindy Williamson: I think when we think about that clear communication and sharing data. It's not just the data that we're sharing. There's you know, contextual information that we can provide. And then the next level is like talking about the process involved, for all of this.

239
00:39:43.540 --> 00:40:12.980
Cindy Williamson: But the sharing of data is is helpful for us. Yes, but also our stakeholders, and that communication really facilitates solid partnerships, I think, across the system and between institutions. So we can work together and collaborate. We can share ideas about processes, projects on some level. We all do many of the same things, regardless of which institution we're at

240
00:40:13.521 --> 00:40:33.760
Cindy Williamson: and then having the conversations that are are beneficial to everyone involved. And then, of course, being able to meet requirements for our creditors. Programmatic, institutional also for T, Hec, and then for the individual campus campuses to be able to provide what system needs to.

241
00:40:34.690 --> 00:40:47.316
Cindy Williamson: I mean, you can go to the next slide, Nick. But I wanted to mention to elizabeth Pemberton is on here. I'm gonna call you out, Elizabeth. But she reached out to us when she saw

242
00:40:47.810 --> 00:41:06.909
Cindy Williamson: that this was going to be one of the sessions for the ie. Summit, and that just like spoke to me. I was like, yes, we will talk about all day about this. So Nick and I met with her, and so she had a little sneak, sneak, peek of what we were going to be talking about. But that's exactly what we want from this right?

243
00:41:07.600 --> 00:41:08.730
Cindy Williamson: so

244
00:41:10.900 --> 00:41:25.129
Cindy Williamson: aligned goals talking about continuous improvement. But that data gives us a way to make informed decisions, to guide our planning, not just seeing what our planning needs to be or where we think we need to go. But

245
00:41:26.410 --> 00:41:47.949
Cindy Williamson: yeah, there's just so much to it and the data that we have in our new dashboards. They're not live yet. But stay tuned but this they absolutely facilitate continuous improvement and individual faculty staff. Again. Department heads. Deans.

246
00:41:48.060 --> 00:42:04.249
Cindy Williamson: vice chancellors, you know, can really leverage the data for enhancement of their programs and services and make sure that they're allocating resources in the appropriate places. And then I'll mention Sacco. See? One more time. But our creditor

247
00:42:04.370 --> 00:42:12.529
Cindy Williamson: wants to know that we're following our processes and that we're we're continuously improving. And I think something like this is exactly

248
00:42:13.620 --> 00:42:19.408
Cindy Williamson: something, you know. One thing that we can use to provide evidence of that continuous improvement.

249
00:42:20.520 --> 00:42:28.813
Cindy Williamson: so yeah, we can provide artifacts. We can provide assessment results. But I think, having them in a way to

250
00:42:29.400 --> 00:42:39.280
Cindy Williamson: a way that someone from outside our system, our campus can look at it and answer their own questions is is super helpful.

251
00:42:43.430 --> 00:43:07.899
Cindy Williamson: And Dick and I also talked about strategic alignment across levels. So yes, we have ut system. We have an overarching strategic vision. We have priorities that have to be well communicated, and then campuses can align with those priorities. So then, down to the campus level, we have our strategic plan.

252
00:43:08.508 --> 00:43:17.791
Cindy Williamson: That is, in alignment with system priorities and then executing our plan and making changes at the campus level.

253
00:43:19.110 --> 00:43:31.267
Cindy Williamson: one more way to show evidence of continuous improvement, but also it gives a place for individual departments, units areas, whether it's colleges or divisions. It gives them a place

254
00:43:31.830 --> 00:43:50.349
Cindy Williamson: to align what they're doing. We we want to see that all the way through our organizational structure. So if we're talking about assessment and accreditation, accreditation, yes, start there. But think about what that means, as you you know, look higher in the organization, or.

255
00:43:50.650 --> 00:43:52.409
Cindy Williamson: you know, up to the system.

256
00:43:53.928 --> 00:44:02.260
Cindy Williamson: Okay, so what's next? We are excited. Oh, well, I am. I didn't ask Nick. If he was excited or not

257
00:44:02.720 --> 00:44:28.620
Cindy Williamson: be, we're going to keep collecting data. Of course we're going to update our dashboards. We will make them live. But as part of that process. We've also talked about a plan for informing campus about the dashboards, so not just putting them out there and saying, Hey, they're here. But you know, holding potentially short information sessions and

258
00:44:29.080 --> 00:44:41.549
Cindy Williamson: talking about how the dashboards can be used, what the you know, specific context around the data. Nick, you want to go on to the next one,

259
00:44:42.270 --> 00:44:50.289
Cindy Williamson: and then we'll continue to collaborate. I've already got some ideas for additional dashboards. Of course we gotta get these out and

260
00:44:50.560 --> 00:44:55.740
Cindy Williamson: get people using him first, st but I I don't think there will be anybody who is.

261
00:44:55.770 --> 00:45:17.009
Cindy Williamson: It's not happy with with what we've come up with. But yeah, making sure that faculty staff students, administrators, everyone understands what they're looking at when they go to these dashboards and we we do already have that dashboards. But at Utc as does system and across the system. But

262
00:45:17.382 --> 00:45:26.317
Cindy Williamson: I think these are different than than what has been expected in the past in the past, and we want to be sure and

263
00:45:27.160 --> 00:45:31.929
Cindy Williamson: have a balance between what we feel like is needed, and then what's wanted?

264
00:45:32.120 --> 00:45:34.710
Cindy Williamson: I think this will be a continuous process.

265
00:45:34.970 --> 00:45:41.938
Cindy Williamson: Once we close the loop it opens right back up. And you know, we just keep going with this. But

266
00:45:43.130 --> 00:46:00.370
Cindy Williamson: initially, I think additional dashboards that we'll think about developing our faculty rating of administrators, employment and placement data super important right now. And we actually finally, Ashley, I'm looking at you finally have a process in place. That

267
00:46:00.490 --> 00:46:01.370
Cindy Williamson: is

268
00:46:02.010 --> 00:46:13.379
Cindy Williamson: most efficient. We we have more and better data, regarding employment and placement than we've ever had. So then, also, as part of that you know, quality assurance funding

269
00:46:13.840 --> 00:46:25.000
Cindy Williamson: several of the things that we've talked about are part of quality assurance, funding, and I think being able to display that so that others can see. That's another one of those things that

270
00:46:25.130 --> 00:46:30.979
Cindy Williamson: just like I've reached about outcomes assessment. These things can't be done in isolation. You know.

271
00:46:31.490 --> 00:46:37.799
Cindy Williamson: campus needs to know about quality assurance funding. Yes, it's part of performance funding, and we know, you know.

272
00:46:38.510 --> 00:46:53.358
Cindy Williamson: money is tied to that right. But even even past that I'm I'm thinking about how different programs score on their program reviews or, you know what our senior exit exam data looks like?

273
00:46:54.460 --> 00:47:14.090
Cindy Williamson: it's just so much institutional satisfaction. I'm not going to go on and on. But there's lots of opportunity there, I think. And this has been a great process so far. And I hope that Nick will keep going with me on this. But I I think we've had a a

274
00:47:14.510 --> 00:47:16.160
Cindy Williamson: fabulous collaboration.

275
00:47:17.060 --> 00:47:17.910
Nick Humensky: I agree.

276
00:47:22.110 --> 00:47:26.999
Cindy Williamson: I think I saw a couple of questions like in the chat, but I haven't gone back in there.

277
00:47:28.390 --> 00:47:44.986
Rachel Borashko: Yeah, thank you so much, Cindy and Nick, that was a great presentation. I really appreciated that. I'll invite anybody to also come off mute. If anybody has any questions that they would like to share with the presenters. We do have a couple in the chat. At least one to everyone, and then a couple sent just to me.

278
00:47:46.130 --> 00:48:02.880
Rachel Borashko: Feel free. If you have any questions also to drop them into the chat. Now you can send them to everybody or to me. I can keep your if you're embarrassed about your question. If you think it's a dumb question, it's not a dumb question, and if you really want to send it just to me, I can keep your identity confidential.

279
00:48:03.230 --> 00:48:06.899
Rachel Borashko: Does anyone want to come off mute to ask any questions.

280
00:48:07.180 --> 00:48:32.890
Ashley Ludewig: I have one. So I used to be faculty at another institution. And so I've seen the faculty side of this data many many times, and getting these isolated reports of my students reviews of the course that are just one at a time. They're not put in any conversation with each other unless you choose to do it. So even on a 1 to one basis like, it's difficult to make use of that data. And so when I look at the scale of this.

281
00:48:32.930 --> 00:48:51.090
Ashley Ludewig: it's just mind blowing, and so awesome. And I'm wondering I know they're not live yet. But have you gotten to do some testing or like Beta, work with, like the faculty and administrators, at least a handful of them who will be using this? And and what do they think? Or do they see the same potential that that you and Nick see in this.

282
00:48:51.887 --> 00:49:03.429
Cindy Williamson: That is coming. We have previewed these dashboards to Provost Hale and so he is certainly on board with this. And yeah, I think

283
00:49:03.540 --> 00:49:13.070
Cindy Williamson: we've gone back and forth with some changes we want to make. And so once we get to a point where we're comfortable. You know, with what we have.

284
00:49:13.690 --> 00:49:26.610
Cindy Williamson: That's an excellent point. Yes, share with faculty department heads, Deans. They're all looking for something different from from each of these dashboards, right? So making sure that it's there and they can access it.

285
00:49:27.450 --> 00:49:29.150
Cindy Williamson: Yeah, that's super important.