Jeri M. Mulrow is the Acting Director for the Bureau of Justice Statistics . BJS's mission is to collect, analyze, and disseminate information on crime, criminal offenders, victims of crime, and the operation of justice systems at all levels of government. BJS is a component of the Office of Justice Programs in the Department of Justice.

(Music plays)

John Bailer : The data used to understand crime, punishment, the justice system is the focus of our program today. There is one federal agency that addresses data from courts, victimization, crime types, law enforcement and efforts to improve the criminal justice system. The federal organization that does this is the Bureau of Justice Statistics in a conversation about work in the BJS will be our focus today on Stats and Stories.

Stats and Stories is a partnership between Miami University's Departments of Statistics and Media, Journalism and Film as well as the American Statistical Association. I'm John Bailer from the Department of Statistics and I'm joined on the panel today by Richard Campbell, Chair of the Department of Media, Journalism and Film.

Our guest today is Jeri Mulrow, Acting Director of the Bureau of Justice Statistics and she'll be joining us today to talk about statistics and justice. So Jeri, can you give us a quick sketch of the scope of work at the BJS?

Jeri Mulrow: Wow yeah well yes.

Bailer: I want to start with a really simple one. Oh, by the way, welcome Jeri, it's a delight to have you join us.

Mulrow: Thank you thank you for having me. It's a delight to be here this afternoon. Wow, the Bureau of Justice Statistics is it's one of the smaller statistical agencies in the federal statistical system but to me it has one of the broadest mandates. As you said in the introduction John, we cover a broad variety of data from victimization to crimes reported to police, through the courts adjudication and prosecution, through corrections and then recidivism studies, who comes back into the system who eventually leaves the system. We cover all of those topics both at the state and local level, at the federal level, at the juvenile level and at the tribal level. So our mandate, it's pretty big and it's pretty broad and in order to do that, BJS runs periodically, anywhere between 50 and 60 different data collections over a period of years to try to measure different aspects across that spectrum of the criminal justice system. That's kind of it in a quick nutshell. I can go into any of those particular areas and describe some of the activities that we do in each of those if that's of interest.

Richard Campbell: First let me ask, Jeri, did you aspire when you were a college student to be the Acting Director of the Bureau of Justice Statistics and if you did not, tell us a little bit about your training and how you got where you are today.

Mulrow: I would say absolutely not. I probably didn't even know about the Bureau of Justice Statistics when I was in college. I was actually a mathematics major in college and during my junior year, I was encouraged to take some applied kinds of courses. One of those courses happened to be the area of statistics and as is usual for first time statistics course takers, it was like I really didn't really like that course very well. But my advisor said go ahead take another course and by the time I got through regression and analysis of variance and design of experiments I said, hey statistics is really cool. I think I want to go into that area. I can work in a variety of different topic areas and so, sort of embarked on that. And still had no idea what federal statistics was; why they were important; where that, you know, that that might even be a career path but went on to graduate school and ended up deciding that I would like to work and not in academia but in some other area and ended up at the National Institute for Standards and Technology in Boulder, working there in one of their statistical engineering divisions. Stayed there for a couple of years while my husband was finishing up but then moved into teaching at Southern Illinois University and said hmm students don't really learn well sometimes. And I think that for me I was more interested in doing something different. I still didn't know what that different was but I applied for a job at the I.R.S. Internal Revenue Service and got a job at the statistics of income division, which is a federal statistical agency and I said hey, this is not such a bad area to be in. One of the things that I liked was I had a lot of interesting practical problems that I could work on and in D.C., we have a very large component of the Washington Statistical Society, that is a component of the American Statistical Association, got involved there and sort of found a community of people that I felt that I could connect with. And so I have stayed in and around the federal statistical system throughout my career. So I started like I said, at IRS, the statistics of income division. And also been at the National Science Foundation their statistical organization called the National Center for Science and Engineering statistics and then about a year and a half ago, the opportunity came up to join the Bureau of Justice Statistics as their principal deputy director and I moved over here. After a couple of months, the director announced he was leaving and I was asked to serve as acting director for the Bureau of Justice Statistics. So, this is a very new area to me. Crime and crime data are very new to me, but of course the statistical aspects of collecting data and analyzing data are, of course, my background.

Campbell: Very good.

Bailer: Awesome. So what was the biggest surprise that you encountered in moving to the Bureau of Justice Statistics? Is there a particular data set or data collection effort through the bureau that people would be surprised to know?

Mulrow: Yeah I think that, just the, well the data, in general to me, are so relevant to what's happening in our communities. So we have data from the National Crime Victimization Survey which collects information from individuals about their victimization. And what we find from that survey is that a huge number of victimizations are not reported to police. And so that's the really interesting aspect of that survey is that it gets sort of this, we call it the dark figure of crime. It's the crimes that are not reported to the police department. Everything else that comes through the system that starts with crime that's reported to police because if you're arrested you go through the court system you get you know go through the prison system or the correctional system. All of that is predicated on the fact that somebody reported a crime and then the police arrested somebody. So this victimization survey actually starts out before that, trying to understand what crimes are occurring and why people may or may not be reporting those crimes to police. I think that's fascinating.

Bailer: Sorry to interrupt. So are there particular types of crimes that are under reported? What's the magnitude of underreporting?

Mulrow: Yes. So there is definitely a variation and the amount of crime that's reported. So you can imagine, motor vehicle theft is reported at very high percentages. Almost, you know, 85...90 percent of motor vehicle theft is reported. That's likely because you have to report your vehicle stolen if you want to get your insurance money for it, right. But other types of crime, so we are estimating from our 2014 survey, that about only 60 percent of robberies are actually reported to the police. And really only about a third of rapes or sexual assaults are reported to police. So you can see there's a big variation there in the types of crimes that are experienced and the types of crimes that people report.

Campbell: Very interesting. One of the things that I always ask our guest is about what journalists get right and do well when they're covering crime statistics or other kinds of statistics, IRS statistics, and what kind of drives you crazy about some reporting that you might see in newspapers or on television when the work that you do is made public? Do you have pet peeves? That's another thing. Do you have any pet peeves about the work that journalists do?

Mulrow: Yeah well, there is a lot of reporting of crime and that's one of the interesting areas of working at BJS is that our data are cited quite often. I think we…a lot of the staff actually have quite a bit of a rapport with various different reporters around because people will...reporters will get interested in a particular topic of crime and then they get connected with one of our staff. And they often will call them ahead of time or when they're writing the stories in order to understand what data are available; how to understand what those data are and how to put that into the context of what's going on. So I think that, for BJS, they have a pretty good record of working with the press in order to sort of mitigate some of those misinterpretations that might occur; taking the data out of context or not really fully understanding what it is that we're reporting on; what this population actually covers and what populations it doesn't actually cover. So, for example in our corrections area, we have information on prisoners and we have information on jail inmates and those are actually very different kinds of populations of people that end up in those two correctional types of facilities. So we work hard to educate and help folks understand how our data should be used and understood but I think that's a pet peeve across people who work with data and present information that that data get taken out of context.

(Music plays)

Bailer: Well you're listening to Stats and Stories where we discuss the statistics behind the stories and the stories behind the statistics. Today our focus is on crime statistics. I'm John Bailer from Miami University's Statistics Department and I'm joined by Richard Campbell from the Department of Media, Journalism and Film. Our special guest today is Jeri Mulrow, Acting director of the Bureau of Justice Statistics. So Jeri, what exactly does a director of a statistical bureau do?

Mulrow: Well since I'm a…we are a very small bureau, it seems like the director does pretty much all sorts of different things. I interact with a variety of different people both within the building so the Bureau of Justice Statistics, it is within the Department of Justice. We do a lot of collaborations with other organizations within the department, such as the Federal Bureau of Investigation or The Office of Victims of Crime...so part of my role is to collaborate and make sure that there are...where there are touch points on data that we're working together and not at odds with each other...so I'd say that's part of my role. Another part of the role as the director is reviewing publications and products that come out; making sure that, you know, they're accurate, that they're timely, that they're relevant, that we're providing information that is, what I would call, policy relevant but policy neutral. One of the roles of the statistical agencies is to provide data for policymakers but not to make that policy ourselves. So you know, our staff are pretty good at that...that making sure that that's occurring, so a piece of that is. Another part of the role is I have a lot of direct interactions with staff because we are very small agency. I have a little under 60 staff so I know everybody on staff. I pretty much know what they're working on, so I have quite a bit of interaction there. And then I would say the fourth piece of it is a strategic planning and where are we going and how do we you know that's sort of the resources allocation; how do we think about where the budget is going; where do we think we need staff; where do we think our staff…what skills and training to our staff need; what kind of data gaps do we have; so more of that broader planning piece of it. So I would say that sort of the more broad areas of where I work into.

Campbell: So this may be a little trickier question but I know how journalist respond, for instance, when during a presidential campaign...crime statistics are used by candidates and sometimes they're not accurate; sometimes they're not right. I know what journalists do. They kick into fact checking mode and I'm wondering what folks at the Justice Statistics bureau do when they hear things like, okay, crime has never been worse or the murder rate is the highest it's been in, you know, 47 years. How do you respond in those instances?

Mulrow: We are not very proactive at responding in those instances. If we're contacted about that by someone, either who wrote the article or someone who is questioning the article, we will provide a response to that. But we do not go out and actively try to respond to every article that's produced and review and correct it. We do not have the resources to do that. I'm not sure that really answered your question.

Campbell: That is a good diplomatic answer and I appreciate it.

Bailer: Well I think that's a fair answer. You're going to be a...you're a resource that anyone can go to.

Mulrow: Right.

Bailer: You made the comment about producing policy relevant data and analyses. Can you give an example of some of the policy relevant...how some of the data analysis that you've done, has been integrated and impacted policy discussions?

Mulrow: Well, let me think. There are, so for example, there are quite a few questions around those held in correctional populations. So, for example, I'll take an example from the previous administration just so it's not, you know, a current day kind of thing. But there are...there were a lot of...there was a lot of discussion about reducing sentence lengths for drug offenders from the last administration and so trying...so...from BJS's role then, we have a lot of data collections from the correctional facility themselves on the inmates. We have some collections on the inmates themselves and so what we would try to do, in that case, is to build in questions in those data collections that would try to get at the percentage of people that are using drugs; that had used drugs; that are in the correctional populations, for example, in jails; what kinds of treatments would they be getting. And then, another way of looking at that would be to historically get some criminal history records for people who had been in those facilities and try to look back at some of the information that they provide on drug use. So those are the kinds of things that we would try to do...is put together pieces from our various different collections to address those questions that are coming up from the point of view that we would have the data. Or we would try to add questions to existing data collections to help to try to provide a picture of the national landscape of what was going on in terms of those questions. That's the kind of... that's the kind of thing that we try to do. We also will do focused reports based on our data that might address a particular topic that has been in the policy route for the last, you know, few months or something like that. We are not the quickest; we're not like journalists where we'll turn it around in a week or two. Our process is a little bit longer and slower than the federal government.

Campbell: How do you deal with issues like, we know for instance, that there's been an uptick in violent crime. I think this was in 2015 but that overall, crime is down. That's kind of...if you're a journalist, that's kind of a hard story to get your head around and explain what's happening there when you see data like that. Can you help me with that a little bit?

Mulrow: Yeah it is a tough one where we are noticing, you know, that definitely crime has been going down over the last probably two decades, if you look at the graphics. There is some increase in, there seem to be some increase in crime in some areas. It's not necessarily...we're not really sure until we get this next year's data and for us from a national point of view, whether that's an overall uptick or whether that's really more of a measurement kind of issue that was going on. Now we do look at the F.B.I. data which is a little bit more granular and at that, when I say granular, I mean at the local and state levels. So they collect data from local jurisdictions; try to follow some of that. We may try to pull in some of that information and look at what's going on with those data too in connection with our overall national level data to try to look at what's going on. Right now, I think we don't have a good picture of what's going on nationally; whether crime is up or not. We do, you know, monitor what's going on to our Crime Victimization Survey too and again, it's unclear right now.

Bailer: Are those very complex surveys that are being conducted as part of the Crime Victimization Survey?

Mulrow: Yeah that is…it is a national household survey. It goes out to about 125- thousand households and maybe 200-thousand people. So we're continuously in the field. Because it's a household survey, it is clustered in particular areas of the country for efficiency, for statistical efficiency, and so. Crime rates are low so when we...we have to actually aggregate quite a few rounds of data in order to be able to talk about some of the levels of crime.

Bailer: That's a good thing, I mean, you know!

Mulrow: Yeah. It is a good thing!

Bailer: I'm just curious about when you were talking about the crime going down over the last two decades and it's always an interesting question about what's the window over which you would look to think about establishing these trends?

Mulrow: Well, we try to provide as much historical data as we actually have. Usually our data is on an annual basis...it's not on a more frequent basis. So we look, we try to look overall what's happening. Some people try to look at, you know, the last five years of the last decade. But given that we have the data, we try to usually present what's happening and then we'll try to talk about it in a more relevant what...how is it compared to last year, to give people that both pictures.

Bailer: So the short term annual trends then. Very good!

(Music plays)

Bailer: You are listening to Stats and Stories and today we're discussing the statistics used for evaluating and understanding crime statistics. So from a for a general audience perspective, you know, how do you best understand the importance of, kind of, the work that's done in your office?

Mulrow: Boy, well. I mean I think overall that the federal statistics...federal statistical system provides a important data infrastructure for the nation overall. The Bureau of Justice Statistics contributes to that data infrastructure by providing national information on a variety of different aspects of the criminal justice system. We provide our data widely; it's widely available for the public researchers, for policymakers, to use in the form of either reports or data files and even for public, or I would call private companies to take that data and then repackage that data. I think that's a pretty common thing that happens with the federal statistics data and I think what happens is the public doesn't really understand or loses where the source data come from. So they think that data are magically generated and that good quality data just sort of comes out of the air but I think that the federal statistical system works really hard to provide that solid core infrastructure that can then be used by a host of people for a variety of reasons. I don't know maybe I lost focus there! Does that answer your question?

Bailer: No…that's very good, thank you.

Campbell: So earlier you mentioned the high rate of reporting motor vehicle crime and the low rate of reporting crime, rape and sexual assault. I think again, from general public's point of view, how do you find out the number of victims who actually aren't reporting rape and sexual assault? How do we know that's only a third of those crimes are reported?

Mulrow: So as part of our National Crime Victimization Survey, we do individual level interviews with people in households and we have quite a extensive survey that goes through asking about the time, types of crimes that they've experienced over the last six months. We ask a variety of questions about those crimes; the perpetrator of the crimes and then we ask, you know, did they report those crimes. I mean we do have questions about whether they reported those crimes to law enforcement; why they didn't report those crimes to law enforcement and then if they did report the crime to law enforcement, you know, what were their experiences with the law enforcement officials. So it covers kind of a variety of questions to try to tease out what crimes are being, you know, committed; what kinds of victims are they and how they're thinking about those crimes. We do a lot of work to develop those questionnaires and what we call kind of testing are going out and testing to be sure that we're getting reliable reporting from a variety of different people. So we spent quite a lot of time working on how to ask those types of questions to get at those kinds of information. It's a bit of a sensitive topic.

Campbell: Yes.

Bailer: Of course.

Mulrow: So we have a series actually of questions and we do...we instruct our interviewers to make sure that the questions that we're asking are not impacting the person who is being interviewed. You know, we make them...we let them know that they can stop at any time. They do not have to answer the questions. These are voluntary surveys. So we work very hard to provide them with information. If they feel like they need to talk with someone else, we often bring materials about where they can seek services if they were victims of crime. We do provide those, sort of, other kinds of activities around that survey.

Bailer: You know you mentioned earlier that part of your responsibility is to think about strategic direction and strategic planning for your agency you know I'm just wondering, what's the future hold for data collection and statistical analysis or modeling at the Bureau of Justice Statistics?

Mulrow: Well a lot of our data collections are based off of administrative information. So we do try to collect data from…there are 18 thousand local law enforcement agencies that we do try to collect information that is in their administrative information system. So I think the way that we collect that data, I think, is going to change. I think with the ability to, you know, maybe process and entail data more efficiently, using technology may allow us to change the ways that we collect some of that administrative information. I think that the way that we're doing that National Crime Victimization Survey is probably not sustainable in the near future. Interviewer administered surveys are very expensive. I think the ability to be able to maintain the costs associated with that is going to be a challenge for us and I think we're already doing research and development work to figure out how we might move that survey to a self administered survey and what that would mean in terms of the sample design and the estimation work and the questionnaire design and all sorts of things like that. I think surveys will be less and less for the federal statistical system overall...because of the cost and the response rates and all sorts of things. I do think that there are challenges to using what we call alternative sources of data, other types of data, that are out there that we haven't been using; questions about the quality of the data; questions about the sustainability of the data. So, for example, if we were to look at using commercial data for something particular or using data from scraping the web. Let's use that for an example. I think the ability to do that is sustainable but I think that the data that's available on the web is not. It changes over time and so may not be a sustainable source in terms of trends over time. So for it, for you know, like measuring...we can take a relevant example, police shootings at those one that's in the newspapers you know. Reporters right now are very interested in reporting on that but. Who knows what the future will be if that will continue to be a hot topic and will continue to cover those in a, you know, full set of coverage we don't know. I'm not sure how sustainable those kinds of data techniques are. But I do think that we have to continue to look forward and figure out how to use some of these re of data collection and you hinted at some of the challenges of merging and collating these types of data that we haven't traditionally been using; sensing data, GPS type data, all sorts of other data sources that now that statistical methods and techniques to analyze, you know, organize, analyze and produce information from. I think we need to look into those. I think that's where the future is going to be.

Bailer: You talked a lot about kind of the future of the data collection and you hinted at some of the challenges of merging and collating these very disparate data sources. Can you talk a little about the models, the statistical models and the prediction... predictive models and how predictive models might be used for analyzing some the data that you guys produce?

Mulrow: Well we haven't really weighed into that area much at BJS so I can't really talk too much about that!

Bailer: Oh no worries! It just sounds like that…

Mulrow: But I think…

Baile r: It's an interesting source that you have that I imagine others will start to consider using that.

Mulrow: And I think it would, I think what we would like to do is partner more with, you know, have more private, public academic partnerships to bring in methods that we know are out there but, you know, we, in house, don't have the staff expertise and or the bandwidth right at the moment to do some of these things. And we're busy producing the information and trying to do just the basic analysis of these data. But there are many more things that can be done with the data...and putting these data together with other types of data I think; environmental data, like what's going on with wages and earnings and where are people living and how are they getting to where they might work and all these things impact what's going on and crime data.

Campbell: An example of that might be... have you done work with the Marshall Project at all which is, you know, Bill Keller, the former editor of The New York Times,is non profit group looking at US social justice issues and criminal justice issues. Is that a partnership that might be might be in the works?

Mulrow: So I am not aware of that at the moment.

Campbell: It is fairly new and it's been in the news because, you know, donations to investigative units on a national basis are up and that's something that's a little that's unusual.

Mulrow: Yeah I know that there are private foundations that are funding a variety of projects in the criminal justice area. But we have not done as much of that partnering as we probably ought to be doing.

Bailer: So if a student wanted to get involved and at some point in their future, work for the Bureau of Justice Statistics, what's the kind of background that would make them, you know, really well positioned to contribute to your shop?

Mulrow: Well I think that the usual statistical methods are a good basis to start with. I think that the data computing data science types of issues also are a good foundation. An interest in criminal justice, in that particular area of criminal justice, is also something that would be useful to have maybe a little bit of background in that...but it's you know there's always a should you hire somebody who's in the subject matter area and then you know, train them in the statistical data science areas or should you hire people with the statistics and data science background and train them in the criminal justice area. I think there's a little bit of both happening. I think all of it can happen and it's just really an interest in wanting to work with federal data at this level. You know, there are some pretty interesting challenges for us from a methodological and statistical point of view of data collection or and/or of analysis there's interesting subject matter...area topics that people can delve into to try to understand what data we have and then help us understand what data gaps we have. So I think there's just a variety of ways to come into it. Mostly, interest.

Bailer: That seems to be a common theme, you know. You're not interested, it's hard to contribute.

(Music plays)

Bailer : Well Jeri, that's all the time we have for our conversation today. Thank you so very much for joining us.

Mulrow: Thank you for having me!

Bailer: Oh it's been great. Stats and Stories is a partnership between Miami University's Departments of Statistics and Media, Journalism and Film and the American Statistical Association. Stay tuned and keep following us on Twitter or iTunes. If you'd like to share your thoughts on our program, send your mail to statsandstories@miamioh.edu and be sure to listen for future editions of Stats and Stories, where we discuss the statistics behind the stories and the stories behind the statistics.