Interview with Julie Volckens, Director of Assessment in the Office of Institutional Equity and Compliance听& Stephanie Foster,听Assessment Lead in the Center for Teaching & Learning - March 2021
Stephanie Foster听
"Hello! We're here today with Julie Volckens. Julie is going to talk with us a bit about the Culture Survey that she has been running here at CU. Julie, why don't you tell us a little bit about yourself and what you do here at CU."
Julie Volckens
"Well, thanks for having me. I'm currently Director of Assessment in the Office of Institutional Equity and Compliance, OIEC. But I've been on campus doing assessment since about 2009 when I started out in Community Health. I then worked in assessment in Housing and Dining before coming to OIEC. And before that I was a social psychologist teaching out in Vermont. I guess I was looking for an opportunity to impact more people and think about things in a larger public health framework rather than an individual-level approach."
鈥婼tephanie Foster
"Wonderful! We're so glad to have you! I've asked you to come talk to us today about the Culture Survey. I understand this is a survey that you have created here at CU with others, and you've rolled out a pilot, and now you're getting ready to roll it out to the campus at large. So we'd love to hear more about that: what's it about, what did you set out to learn and how's it going."
Julie Volckens
"So听in 2014 the campus, as it had done--for I think a couple of decades--the campus assessed undergraduate and graduate students with the Social Climate Survey. And from that survey, across administrations, we were able to see that there was a disparity in the experiences that students were having. Students of Color, LGBTQ+ (Lesbian, Gay, Bisexual, Transgender, and Queer or Questioning and others) students, and students with disabilities were having a less positive experience. That got us thinking about how we could do something more comprehensive with the campus including looking at the impact of other aspects of identity and broadening the scope to staff and baby直播app.听
Through analysis, we identified those parts of the survey that were giving us the richest information. For instance, we found that 2014 first-year students鈥� sense of belonging, which includes feeling respected, supported, and valued, was a significant predictor of the likelihood that they would be retained at CU until the听 spring of their junior year. We also found that between 2014 and 2017, we were losing LGBTQ+ graduate students at a higher rate than non LGBTQ+ students; they were leaving before completing their degree or program at twice the rate and a diminished sense of belonging was an important factor in not retaining them.听
We wondered how we could make that survey even better, and then more importantly, how we could extend that to everyone by using the 2014 surveys as the basis for assessing baby直播app and staff as well. So we spent the next three years working on that, identifying the best questions from the surveys that were administered in 2014, adapting them to make them appropriate for the different contexts and experiences of staff and baby直播app roles, and piloting those questions in collaboration with different units on campus.
We听drew from our years of work with students and employees, and from feedback provided by the students, staff, and baby直播app who worked closely with us to develop the survey, especially the LASP Climate Committee. Especially important to us was making sure that every question would be actionable. We also drew inspiration from surveys administered at other universities and research institutions, finally landing on a set of questions that we first piloted at LASP (Laboratory for Atmospheric and Space Physics). LASP was willing to be our inaugural site and with that administration, we were able to provide LASP with really usable data. With that LASP data, we used factor analyses and statistical modeling to pare the questions down by about half. We were then able to pilot the revised version at different departments on campus. In fall 2019, the survey was adopted as part of the Academic Review and Planning (ARPAC) process to inform the self study.
Beginning the week of October 18, all students, staff, and baby直播app will receive an invitation to participate in the Campus Culture Survey (CCS). We had planned to administer the survey last year, but we decided to put it on pause during the pandemic. In addition to supporting the ARPAC process, the CCS will serve as an important source for benchmark data for the IDEA plan and will be one way to measure our progress going forward as we reassess every 4 years."
鈥婼tephanie Foster
"So听in the pilot process you piloted this new survey with several different departments. What were a couple of the most important things that you learned from that process?"
Julie Volckens
"Well, one thing that we didn't expect when we launched was how important people's experiences of hostile or negative treatment were to understanding how students, staff, and baby直播app are experiencing the campus culture. When we were determining how to measure hostile behavior we looked to the large body of research on baby直播app incivility. We know that 鈥渋ncivility鈥� is not an unproblematic term and we don鈥檛 use it in the survey--what we鈥檙e trying to understand are negative behaviors that undermine the work culture. These behaviors might also be called bullying or harassing. When people attribute this type of behavior to an aspect of their identity, that could meet the definition for a microaggression, and could possibly represent a violation of the discrimination and harassment policy. So we were really surprised at how many people reported experiencing this type of hostile treatment and then how much those negative experiences were shaping people's overall experience of the baby直播app culture.听
Having examples of the specific problematic behaviors that are occuring in a baby直播app provides a roadmap for what departments need to address to improve their culture. For instance, one of the most common examples of baby直播app hostility is being constantly interrupted or talked over. That's very actionable behavior. Units can make agreements about how to reduce and eliminate this problem, and how they will respond and repair when these types of situations happen going forward.
The other thing that we found very surprising was how many people perceive that department leaders, whether it's baby直播app, chairs, supervisors, or directors, need more skills to address problematic behaviors that undermine the work environment. The survey has uncovered an area of great need that can be comprehensively addressed; people can get more skills to manage problematic behaviors, and also people can get better at anticipating these kinds of problems and putting measures in place to reduce the likelihood of these harms happening in the first place."
鈥婼tephanie Foster
"So in this process -- you've been working with for quite some time now -- what's your favorite part?"
Julie Volckens
"I guess my favorite part, and I think anyone who does assessment would say this, is the data analysis, where you get to understand the dynamics of a unit; what are the strengths that make them shine, and where they are struggling? Honestly, no two departments are alike. Strengths vary, concerns vary. So I find it super satisfying. I work with this incredible team and together, we put together a report that helps units understand where they are, where they could go, and options for getting there."
鈥婼tephanie Foster听
"So what will be the next steps as this survey rolls out this fall?"
Julie Volckens
"Since the late summer, we鈥檝e been communicating to the campus that the survey is coming, including what the survey will be able to tell us about our strengths (at the campus, division, and department levels) as well as where we have work to do. Since 2019,听 we've been extraordinarily successful at getting a high response rate, more than 50% of units responding to the survey. That's been really exciting. The way we've been able to do this is through our outreach efforts--really helping people understand what the survey measures and how it can be used to support change and accountability. There's a lot of enthusiasm on campus for improving the campus culture, and for understanding what we鈥檙e doing well and where we need to grow. This includes understanding how different groups may be having less positive experiences. We have been working closely with our key campus stakeholders and meeting with groups across campus to share information about the survey, including past findings and recent efforts that have been undertaken in response to the findings that have led to meaningful change. The launch of the official campaign and announcement from the Chancellor, Provost, and Chief Operating Officer will start in early October. There will also be a coordinated print, bus ad, and social media campaign. The survey will launch the week of October 18 and members of the CU community will receive an invitation in their CU email to participate. Students will receive a $7 incentive for participating. We have more information about the survey and a list of FAQs on our webpage, including how we are protecting the privacy and anonymity of our participants."听
鈥婼tephanie Foster
"What would be a really good result from all of this work?"
Julie Volckens
听"A really good result would be having a robust rate of participation that would yield an irresistible call to action and provide a roadmap for change and accountability. We need to hear every voice. We already know that the survey findings can be used to implement practices that can improve baby直播app culture. We worked with one unit whose results showed that they had issues with supervisors and department leaders needing more skills for addressing problematic behaviors that undermine the baby直播app; this included about 40% of the department indicating they had experienced hostile baby直播app behaviors. They invested in a strategic plan that included capacitating supervisors and department leaders with more skills for supervision, in conducting more inclusive hiring searches, and in building skills for having difficult conversations and being effective bystanders. After 18 months, they had cut their rate of incivility by half, and saw statistically significant improvements in sense of belonging, perception of supervisor skills, and feeling respected by colleagues. So units can make big changes as long as they understand where the the points of entry are."
References:
听2021 Campus Culture Survey -- Information, Timeline, Security & Privacy
听Microaggressions -听OIEC
Workplace Incivility
Workplace incivility is defined as, 鈥渓ow-intensity behavior with ambiguous intent to harm the target, in violation of baby直播app norms for mutual respect; uncivil behaviors are characteristically rude and discourteous, displaying a lack of regard for others鈥� (Andersson and Pearson, 1999, p. 457).
These behaviors frequently lead to an unfriendly and unproductive work/academic environment that has consequences for employees鈥� loyalty, retention, productivity, health, and work quality. When targets attribute incivility directly or ambiguously to identity, these acts characterize microaggressions, and possibly represent conduct prohibited by the discrimination and harassment policy. A review of the research on the consequences of baby直播app incivility is available in a 2019 OIEC literature review.