Episode 24: Reading, writing, and enrollment analytics
Release Date: 01/18/2017
Michael Kabbaz is Vice President , Division of Enrollment Management and Student Success at Miami University. His Division includes Offices of Admission, Bursar, Career Services, Enrollment Communication, Enrollment Operations and One Stop Services, Enrollment Research and Analysis, University Registrar,Student Financial Assistance and the Student Success Center.
Rosemary Pennington: Millions of young adults will start their lives as college freshmen fall 2017, but before they start shopping for dorm room supplies and planning their class schedules, they have to apply for admission first. Application deadlines at American colleges and universities generally fall between January 1 and February 1. Applying for college can be stressful as high schoolers try to figure out how best to present themselves, and whether they have the right mix of grades and extra-curricular to get into their first choice school. Today on Stats and Stories we get the inside scoop on college admissions, enrollment and retention. Stats and Stories is a partnership between Miami University’s departments of Statistics and Media Journalism and Film and the American Statistical Association. I’m Rosemary Pennington, our regular panelists are Department of Statistics Chair, John Bailer, and Department of Media, Journalism and Film chair, Richard Campbell. Our special guest today is Michael Kabbaz, Vice president for enrollment management and student success at Miami University. Thank you for being here today Michael. To start off our conversation I thought I’d ask you to explain simply what your job at Miami entails, and maybe how data plays into that?
Michael Kabbaz: Yeah, so, my job is to oversee the student life cycle, from the point of recruitment all the way through student success. So, the idea is that what type of students we recruit, how we attract them, how we support them while they’re here and ensure that they get great jobs and go to great graduate schools after they leave, so it’s an all-encompassing student life cycle approach to enrollment management here. One of the things, data undergirds each part of that process, from the types of students we recruit, to the types of high schools that we visit, to the types of students we admit, and then how we support them while they’re here, and certainly we care greatly about the outcome of those students in terms of their employment once they leave the university.
John Bailer: So how much has your business changed over two decades worth of time?
Kabbaz: Yeah, you know, I think over the last fifteen years we’ve gone from a very qualitative profession to a profession that is very much driven by data, the idea that, you know, the qualitative aspects of understanding the student, it’s certainly a one-to-one understanding of the student’s application, their interest in the institution certainly plays into that, but now, how many students do you admit? How many, you know, what division do they fall into academically in terms of their interests, how many students should you admit to the business school, how many students should you admit in the engineering area? So that the whole idea is that students, when you hit nearly 30,000 applications, qualitative is great but you have to have the quantitative to undergird that process.
Richard Campbell: Michael what do you think about these ranking processes that different places use? I know that we probably got a really nice boost when US News and World Report started ranking us really highly on commitment to Undergrad education, how much did that help Miami? How much did it help in your job? Or did it at all?
Kabbaz: Yeah, you know what’s interesting, rankings, in fact, interestingly enough I was looking at some recent research on this, and rankings, interestingly enough, are more of a decision maker as it relates to what colleges students apply to for the more selective student. So the idea that the types of students that Miami is trying to attract, actually rankings do, in particular when we talk about rankings certainly US News is the one top of mind, I think every place is getting into the rankings business lately, and we can talk about that certainly. But what’s interesting, the US News in particular, when you start to look at the types of things they weight, some of those things involve faculty resources, some of those things when you start to look at the outcomes related to this, medical schools within institutions tend to fair well because of the resources, but what’s interesting about the Undergraduate teaching ranking is that we believe very much that that’s what Miami stands for as a very focused undergraduate institution, so that has helped greatly in terms of the market. In particular international students care greatly, in China, is something, an example of the interpretation of US News in China is they believe it’s a government ranking, so institutions that tend to be ranked highly certainly have an opportunity to attract international students as well.
Campbell: Are there particular data points in that ranking that we do well on, we must do well on some things that shoot us near the top, that maybe other public institutions aren’t doing. I think a number of people, I wasn’t surprised by that data, but a lot of people were.
Kabbaz: Yeah, so let me answer that, the two areas where that actually works against us is the faculty resources and our endowment for the type of institution we are, however, on the flip side of that, we do really well on the prediction around … they actually do a predicted graduation rate and Miami over predicts what they would calculate, so we actually do pretty well in terms of that, the outcomes of our students are strong. And certainly the inputs around our student selectivity, the quality of the profile is definitely a strong position for us. The other thing that is an important part of this is reputation plays into this, they do reputation, and they have high school guidance counselors’ reputation, a qualitative assessment as well as pure institutions.
Bailer: You know, you’re hitting on a bunch of the subjects or the factors that are drivers for an external ranking scheme of different colleges, so you’re talking about a derived measure that students might use as one of the input factors. One of the questions that I’m sure you must hear very often are, what are the things that are really important for driving the likelihood of admission success to a university? How does that play out differently at different institutions?
Kabbaz: Yeah so, one of things that’s really interesting about this topic in terms of student decision making, is back in the early ‘70’s out of UCLA there was a CIRP survey, and when you look at the CIRP survey, and in fact they annually look at about 150,000 students, so it’s a large and…going back to the early ‘70’s, and actually through last year, the number one reason that students select an institution is academic reputation. That has been the number one factor and that hasn’t changed. Number two: do I get a good job? And increasingly, is financial aid available? So, and we get into a net price conversation, so I think as you start to look nationally at the landscape of higher education, certainly one of the things that people will say, higher education is getting more and more expensive, and even for families that have affluency, price is becoming much more of a factor, but those factors have always been in the place, and we can certainly localize it to some of the conversations around that. But, academic reputation has been the driver.
Bailer: So those are the predictors of whether or not someone is going to apply to an institution, so then the question from the student’s perspective is, what are the factors that I possess that’s going to help drive the institution’s decision to admit me?
Kabbaz: Ok so, you’re talking about the qualitative factors, or the quantitative factors? A variety of things?
Bailer: Yeah so what plays into that?
Kabbaz: So, I think what’s really important is you look at the national landscape of institutions and the vast majority of institutions are open enrollment or near open enrollment, so I think when we have this conversation if you look at the 4000 institutions across the country, the vast majority have few, if any selective criteria, so what you start to move into are the hundreds of institutions that apply a selective, in terms of admissions, so competitive admission we’ll say. And those are going to certainly be number one factor will be grades and college prep courses, overall grades, test scores are going to be in that overall, and those are going to be the driving factors. Essays that the student writes, college recommendations from the guidance counselors, teacher recommendations are going to play into that, and what I would call other factors. These are the ‘grit’ factors that we talk about, what are the types of things that the student has done while they were in school, did they overcome anything in terms of showing their grit. The other thing is certainly the activities that the student was involved in in high school. So very few institutions when you look at the entire landscape, probably 10% of institutions are employing those types, versus a more open enrollment or very slightly selective institution.
Campbell: A question comes up in terms of how journalism covers this whole process, and I know you have experience talking to journalists, both student journalists here and elsewhere. Do you see instances where journalists are kind of looking in the wrong places or emphasizing the wrong things? Do you have some pet peeves how you feel like you’re represented when you talk to journalists about some of these issues?
Kabbaz: You know, it’s funny that you use the term pet peeve because that was exactly what was on my mind as you were asking the question. Loan debt is an example of something that is very very mischaracterized in the media. When you hear that loan debt… the example I always say is, I say that the New York Times every year is going to produce that student who graduated from a premier institution with over 100,000 dollar debt, and they’re living in the basement of their parents. So the media tends to use that as way, the lens to look at. The other thing related to that conversation about loan debt is the one that the media really gets on is that they say that the credit card consumer debt has outpaced that of loan debt. So the example I always use on this is yes, the type of education is going to give you a lifelong opportunity in your career versus that vacation that you couldn’t afford over in Hawaii. And that’s the kind of comparison that the media often… they make a very complicated argument simplistic in the way in which they do it. When you drill down into the data and you look at the national debt, or the debt associated with students, what you will find is many of the numbers we’re talking about are extreme outliers. Many are students who have accumulated a lot of debt as it relates to medical or law school. But again there are certain times when students borrow too much money, but the media tends to definitely make it a bigger issue than in reality.
Pennington: You’re listening to Stats and Stories where we discuss the statistics behind the stories and the stories behind the statistics. The topic today, college admissions, enrollment and retention. I’m Rosemary Pennington, joining me are panelists, Miami University Statistics Department chair, John Bailer and Media, Journalism and Film Department chair, Richard Campbell. Our special guest is Miami University vice president for enrollment management and student success, Michael Kabbaz. As you’re talking about what factors into student admission, I was reading some stories today about this, I don’t know if you can call it a trend after a couple of years, but there seems to be some suggestion that enrollment at universities nationwide might to be going down a bit, sort of from a high from a few years ago. Do you have any sort of thinking on why maybe this dip exists? Or is this another story that maybe is getting overblown by journalists and we just kind of need to hang back.
Kabbaz: Yeah so, I think part of it…the way to interpret this is the demographics are shifting very quickly, and I think less about the overall number, which basically has a little ebb and flow to it, but when you look nationally, that’s the conversation that tends to be the focus. But when you start to dig into the data, the Midwest is decreasing, the Northeast is decreasing, when you start to look at the west and the southwest, it is increasing at very very high rates, but is increasing with the types of students who are likely to need more financial aid and very diverse students. So it really speaks to…it really talks about the types of institutions…what it’s going to take for us to support those types of students that speaks to national policy around the Pell Grant, about institutional aid focused on need based aid. The types of support services that you need to support students who come from different backgrounds. So I think the picture is much more complicated than the overarching national numbers and it’s very much regional because the vast majority of institutions, even some to the institutions that people will perceive are national institutions, there’s a handful of national institutions, I mean what’s interesting, you take Harvard for instance, and a large share of their enrollment comes from the Northeast. So, this whole idea that a vast majority of institutions draw nationally, that are like a Miami, but much of their backyard is actually part of the evaluation of being able to look at the geography.
Campbell: On that point, Michael, our out of state enrollment at Miami has increased fairly dramatically over the last few years. How do you think about that in terms of being in the Midwest, how do we compete? We’re not the research one institution in the state, Ohio State is, yet I think we have a higher out of state percentage in terms of enrollment. So, how did that start playing into this, and I think this became fairly important after the economic crises, that we just had to broaden out our base for who we were admitting to Miami.
Kabbaz: There’s a number of factors that play into that, first is the demographics are not helpful to us. If you look at the high school graduates in the state of Ohio, it’s roughly 130,000 high school graduates a year, and in many cases declining. African-Americans are decreasing over the next five to seven years. You look at Hispanic students, there’s an increase there. So, the state of Ohio, when you look at the landscape of public higher education, there’s fourteen four year public institutions, there are over a hundred private institutions. It is ultra-competitive in a state where the enrollment is stagnant. So for a place like a Miami, or even to that extent, like an Ohio State, we have the ability to go out and draw from a larger area. So the focus has been to recruit and enroll the best and the brightest from the state of Ohio, but at the same time extend our reach. One, it’s an urgency because of the demographics, but two, it’s also connected to the fact that we can’t meet our mission of the university if we were to just recruit students from the state of Ohio. There’s a financial aspect to that certainly, but also two, in terms of being able to diversify in all senses of the word the enrollment at Miami. Interestingly enough now 65% of our applications last year came from outside the state of Ohio. I think when you look at the demographic trends that’s not something that will change back, in the days when it was much more Ohio than not.
Bailer: So you start each year with a target. ‘I want to have this many thousands of students in a first year class.’ How do you decide, when you start with that kind of number and you know that you have a certain mix that you want to achieve in certain divisions, how do you decide how many applicants you’d like to get? Not you, I’m not putting this all on you Michael, but for an institution that’s trying to plan for this, how do you make a decision, how much do I use historical information, how reliable is that given a system that may be changing pretty dramatically from one year to the next?
Kabbaz: Yeah, I definitely start off with the premise that all models are wrong, some are useful. That’s something we’ve heard before, and I share that with the president and with the board of trustees but often times that doesn’t go very far. What you bring up is a really important topic, and really the focus of what I would consider an enrollment management function at a university. First it starts with engaging in conversations with each of the academic areas about what is your capacity, what types of students do you want to attract, what types of areas of emphasis do you want, and that’s how you build your recruitment program. And then once the students apply to the institution, one of the things that you’re doing, is your deciding how many students to admit to the university, but also how many students do you admit in the business school, how many students do you need to admit in the honors program, how many university scholars do you take in each of the thirteen areas that we have scholars programs. So the way to think about this is, there’s a bunch of models within the models and how we admit them. The interesting thing about this too, a few things have taken place that have made this much more complicated over the past number of years. One, students are applying to more institutions, and in particular the most selective and competitive institutions, so when you look at our competitive set, we last year alone lost six hundred students that were admitted to Miami, went to other places, and certainly when you look at the demographics many are big 10 institutions, you have your Vanderbilt’s, you have really strong private institutions that are competing for the types of students we’re competing for. So, the models have continued over a period of time, it’s using history to predict the future, and to me there’s always this art and science in terms of here’s what the model says, but you know, that doesn’t feel right to me, or I think the model’s underpredicting that. So, certainly that’s played more complexity in this, because you have to put all of those pieces together because at the end of the day you can’t have more than 3800 student enroll or else they’re not going to have housing. So there are real constraints and then there are constraints that faculty thinks the class sizes get too big. So there’s real constraints and then perceived and otherwise constraints.
Bailer: So if I could just quickly follow up, so do you evaluate your models each year, and look at the performance and go ‘Oh man, did we miss on that one’ and then try to analyze what are the components of that, can you share a story of one where you realized some other predictor that really emerged as being critically important that you had not previously considered.
Kabbaz: Yeah, you know, oftentimes, and it’s probably not unlike Miami, there’s a handful of factors that are going to be the biggest drivers of the model in terms of predicting enrollment, because what you’re trying to do is if you admit X number of students, how many of those students are going to come. And biggest drivers are academic ability of the student, which won’t be a surprise, students who are academically talented, and we can talk about what that means, it’s ACT, it’s GPA, it’s also, what I would consider and academic index for looking at the quality of their high school curriculum, amount of money offered, distance to the institution plays a factor of that, also too, there’s very strong differences when you start looking at the division of major. So for instance, a Farmers School of Business student is going to have more national draw, more competition to it, so there’s all those different factors that play into it. One of the things that just happened this past year, and one of the best predictors of whether a student was going to enroll at your institution was actually based of FAFSA information. FAFSA, when you fill out the federal financial aid form, families have the ability to put, in rank order, what institution that they want. Where students put the institution was one of the best predictors. This past year the Department of Education thought it was being misused and completely pulled it. So we go from something that is one of the strongest predicators to not at all being able to be used in the model. So we had to have a lot of conversations about what parts of the model should we rely on more heavily. That was a curveball.
Pennington: You’re listening to Stats and Stories and our discussion today focuses on university enrollment, admissions and retention. Our guest is Michael Kabbaz, Vice president for enrollment management and student success at Miami University. I’m Rosemary Pennington along with our regular panelists Miami University Media, Journalism and Film chair Richard Campbell and Statistics Department chair, John Bailer. I have a question that’s more about how you keep the students here. I was reading something today that said Miami’s retention rate is at 92%, so I was wondering if you could explain what exactly you’re factoring into that retention rate and what is keeping students here?
Kabbaz: Yeah so, retention rate…when you look at…a lot of conversation about what does a retention rate mean? There’s a federal definition of first time/full time students who begin in the fall and are here the following fall, so that’s what makes up the 92%, so theoretically we can go and benchmark ourselves against institutions regionally and nationally to have that comparison. That retention rate puts us in the top handful of institutions in the country and the associated graduation rate that is also high here as well. One of the things…a couple factors play into retention models, first and foremost is the student’s academic ability that is going to be one of the strongest predictors of their ability to be here, we have a very strong incoming student body which contributes to it. Being a residential campus is also a strong factor that the students have to make an intentional decision to come to Oxford, Ohio and move into the residence hall. So by the very nature of our population, it’s not a transient population, you have to be able to come to Oxford to engage and 98% of our students live within two miles of campus, so part of it is a high ability student body, one you have a residential and also too, and not just because I have faculty sitting around the table with me, but certainly what happens in the classroom, that faculty have the biggest impact on students progressing in the process. So those factors play very much into why I believe we have a strong retention rate.
Bailer: I’m curious, I’m glad you opened up this…the idea of once a student is here what might happen in their lives? Part of the student success component of your office, and maybe many universities are things like enrollment analytics, the idea that once someone is here there are these student success factors that you’re going to try to track in order to try and predict their likelihood, or maybe to monitor a student who’s at risk of problems. How are models being used there, and what are some of the insights that you think those will provide?
Kabbaz: Yeah so, this is the topic, certainly of the day, as it relates to this work. You have institutions who are very, they would use this term intentionally, intrusive advising, where these models are actually predicting that a student is likely not to be successful at this institution and then becomes a very prescriptive intervention process. There’s actually institutions, very well regarded institutions where if a student is off track for too long they’ll actually work with them to change their major. The Miami philosophy tends to be a lot more let the student figure out what they want to do. So this is an interesting conversation for us, in particular at Miami, so there’s a lot of national conversations to look at, we certainly have to tools and the software if you will to be able to do predictive analytics, where we can get to the fact… in fact this past year we were able to predict three out of four students who didn’t come back we were able to predict three out of the four, and that’s just the academic data. So the conversation becomes, what other factors should we include in that conversation, that doesn’t include any of the co-curricular data, is the student in the library, look at the judicial data. So what you start to get into is this whole conversation…the proverbial big brother. And, we know a lot about a student, if you can connect those pieces together it can be very predictive and I think part of this too, is this conversation, tends to be, particularly from the academic advising perspective is, you can be an academic advisor and very much the way you have done your work is on the qualitative experience. That experience is real, you’ve worked with students for twenty five years and you know if they’re doing this, but now data starts to enter in the conversation and then what do you do with that. And the data tends to create fear of the data being used against the students, so I think that conversation is kind of where we are in that conversation in that spectrum of big data.
Campbell: Following up on that, you mentioned earlier the CIRP data, the freshmen survey data, and if you look at Miami’s data going back, I think this started in the early ‘70’s, we’ve consistently recruited classes that are a little bit more conservative than our rivals, where most public institutions who are our peers more students identify as liberal, here it’s flipped. We also have a high percentage of Catholics compared to other institutions, Catholic population, you now it’s eight, nine, ten percent higher than what you see at other institutions, is this something that you look at in the data as you look at this CIRP data going back that plays…and it’s fairly consistent over time, I mean this has been true, particularly in the political area it’s been true for many years.
Kabbaz: So you know what’s fascinating about this concept is that when you’re out around the country and you’re talking to prospective families and students I very much believe in this concept of birds of a feather flock together, so what you tend to find…for instance about a third of the incoming students have either a brother or sister who went here, mother or father, or a grandparent, so what tends to be the case is that you tend to replicate the student body coming from the similar family background. So much of that is built into the fabric of…but what’s interesting about that conversation is that you start to venture out and start to recruit a much more national and international population, we should start to see those trends shift and the conversation becomes…you know those are things you don’t go out and say…particularly as a public institution there’s no thumb on the scale for the Catholics or otherwise, but you do bring into this conversation as you start to expand our reach around the country, you know, how open we are to other religions, how open are we to the other diversity…as the population shifts in a variety of different ways.
Campbell: I’m always interested in the stories we’re not telling, you know our journalism students aren’t telling. Do you have stories that are surprises to you, things that you wish journalists would ask you? You’re kind of missing a story here, there’s…some of these things that we find over time are interesting to me and I always think, there’s a lot of untold stories in this whole process.
Kabbaz: So even in particularly at Miami or nationally?
Campbell: Nationally or at Miami.
Kabbaz: Well, I’ll start with at Miami, one of the areas that we’ve been spending a lot of time on related to the student success conversation, I’ve put in place a student success center. And the way best to think about these four staff people…they’re not in student affairs and they’re not in academic affairs, what they’re job is to do is to work with students because oftentimes student’s issues are both academic and socio-emotional, right, it doesn’t tend to be one or the other, and through the course of this, I think what a lot of people perceive as Miami, perception right or wrong or indifferent, about affluence, about students, you know…and what you start to dig in we have significant cases of food insecurity, students who do really worry about…and these are real students on our campus, and we’ve had cases of homelessness on this campus. So I think that there’s a lot more to what tends to be when you look at actual demographics, it tends to look like an affluent population from this, but I think there’s a lot more stories behind those scenes than a lot of people perceive Miami to be. In fact the people that work on campus, I think are surprised to hear these conversations.
Bailer: You know you had mentioned in passing the idea of kind of the value of having all of this data available but also the potential concerns about privacy and intrusiveness that might be associated with it. So can you talk a little about how you think about, and how your office thinks about data and the security and the confidentiality of this information, and sort of what are some the constraints that you operate with?
Kabbaz: Yeah so, first and foremost confidentiality has to be the baseline for everything, and what I view our role is relative to the campus community is for us to share, these are the data we have, this is the information we glean from it, and what is the campus’s interest in going down this route, so I think that the conversation is… and I guess the premise I have, and as I talk to academic advisors around campus or even faculty is let’s not assume that data should be used for evil, let’s assume that data should be used for good. And if you start to use that lens to think about this, what you start to find is, students who are predicted to not be successful, what’s the worst case scenario that if a faculty member were to reach out, or if a student affairs professional were to reach out and just to check in on how that student’s doing, the worst case scenario is that that student’s doing fine, or the alternative is that that student could actually use some help. So the idea is, I think we have to reframe the conversation, is that data can be used negatively, I think we have professionals and people who can do that but the idea that we should use it to help students. In fact I would probably push it a step further… I would make the argument that because we know things about students it’s almost incumbent upon us to take action otherwise. I always tend to see it from a pretty data heavy perspective, but also to pressing that against what has been a pretty qualitative profession, meaning broadly defined academic advising.
Pennington: Well, Michael Kabbaz, thank you so much for being here. Michael Kabbaz is the vice president for enrollment management and student success at Miami University. That’s all the time we have for this episode of Stats and Stories which is a partnership between the departments of Media Journalism and Film and Statistics here at Miami, and the American Statistical Association. With the start of the New Year we’re excited to announce this new partnership and a more frequent release of programming, stay tuned and keep following us on Twitter or iTunes. If you’d like to share your thoughts on our programs send your email to statsandstories @miamioh.edu. And be sure to listen for future editions of Stats and Stories where we discuss the statistics behind the stories and the stories behind the statistics.
Click to close the script.