Innovative technology is at the heart of how Trevor supports LGBTQ youth in crisis every day. Trevor’s Technology team is transforming the organization’s life-saving crisis services, increasing its capacity to serve exponentially more youth each year. That includes AI innovations in the organization’s crisis services delivery model, moderating TrevorSpace to keep youth safe, and building best in class systems for onboarding, training, management, and retention of thousands of volunteer crisis counselors.
Tech’s Impact @ Trevor
For The Trevor Project, technological advancements mean being able to serve more youth in crisis, at a faster rate, with a consistently high quality of care.
Since 2017, The Trevor Project has increased the number of digital crisis counselors trained from 100 per year to 100 per month.
Since 2018, The Trevor Project directly served 50% more crisis contacts year-over-year, going from 67K (2018), to 98K (2019), to 150K (2020).
We’re proud to have over thirty full time staff members across product development, AI, engineering and technology operations — and we are continuing to hire across our AI, engineering, UX, and technology departments.
The safety and privacy of the youth we serve is always our top priority. That’s why responsible innovation is core to the values of our Technology Team. And, as our AI initiatives grow, we have a responsibility to create a values-based system to guide how we use and develop AI.
Consulting industry experts: In developing an ethical AI framework to exist within our organization’s mission and values, we have drawn insights from internal and external domain experts to ensure we promote the fairness and equal treatment of all people.
Reducing model bias: Trevor prioritizes the development of models that do not demonstrate unfair bias that impacts people based on their ethnicity, sexual orientation, gender identity, race, or the intersection of these identities. These steps help ensure that every youth who reaches out to Trevor receives fair and equal assessments and support.
Ongoing evaluation: We work across teams with distinct functional expertise to review AI performance and evaluate our models for biases to identify if and what action needs to occur.
We continue to look to Berkman Klein Center, Microsoft, and Google’s AI principles as we progress in building out our own AI and data product strategy and principles.
Help Us to Continue Innovating
Innovation and growth are crucial for us to reach the more than 1.8 million LGBTQ young people in need in the United States.
Trevor’s Technology Department’s purpose is to support and empower the crucial work of everyone on Team Trevor as well as our army of Volunteer Crisis Counselors.
Our work would not be possible without substantial support from our corporate partners. We’ve built scalable, impactful tech solutions with organizations such as AT&T, Google.org, the PwC Charitable Foundation, and Twilio.org. The work from these partnerships prepared us to respond to COVID quickly so that our service remained uninterrupted. In preparation for physical distancing, our organization underwent a rapid, massive tech transformation to move our physical Lifeline call centers in NYC and LA to completely remote operations for the first time ever to ensure our crisis services remained available for LGBTQ youth in crisis.
Case Study:
Using machine learning to assess LGBTQ Youth Suicide Risk with Google.org
The Google.org Fellowship is a pro-bono program that matches teams of Googlers with Google.org grantees for up to six months to work full-time on technical projects. To learn more about the technical details and how the Fellowship empowered us to have a greater impact in a much shorter period of time read the case study above.
Visual Overview of Trevor’s Machine Learning Chat Analysis
There are 1.8M LGBTQ youth in crisis each year in the U.S. and Trevor has an ambitious plan to support every single one by 2023. We welcome the support of individual donors as well as Institutional Partners interested in getting involved at a greater level such as through providing funding, pro-bono support, or a combination of both.
The Trevor Project is a Google AI Impact Challenge grantee, and has received $2.7 million in grants and two cohorts of Google.org Fellows. Fellows are Google employees that work for Trevor full-time for 6 months. This support has enabled us to incorporate natural language processing into our platforms as a means to assess suicide risk more quickly.
THE CHALLENGE
There are high-volume times when more youth reach out to us on TrevorText and TrevorChat, causing a shortage of counselors and increasing wait times for youth to receive help. In these situations, ideally, youth would be connected with counselors based on their relative risk of attempting suicide, so that those who are at imminent risk of harm would be connected earlier.
THE SOLUTION
With the help of our Google.org Fellows we used natural language processing (NLP) to train an ALBERT model to learn which types of responses on the intake form were most likely linked to a particular risk level.
This was accomplished through framing the problem as a binary text classification problem, sorting responses into two different groups. The inputs are answers to questions on the intake form that youth complete when they reach out:
Have you attempted suicide before? Yes / No
Do you have thoughts of suicide? Yes / No
How upset are you? [multiple choice]
What’s going on? [free text input]
The output is a binary classification: whether to place the youth in the standard queue or a priority queue. While some specific words or phrases are known to correlate with high risk, the NLP model interprets the entire sentence to determine risk level. This way we understand the entire context of the situation based on the intake form when we determine risk. As counselors become available, they connect with youth from the priority queue before youth from the standard queue.
DATASET PROTECTION & FAIRNESS
Because of the extremely sensitive nature of the dataset, special measures were taken to limit its storage, access, and processing. We automatically scrubbed and replaced data with personally-identifiable information (PII) such as names and locations with placeholder strings such as “[PERSON_NAME]” or “[LOCATION]”. This means the models were not trained using PII.
The model was also specifically trained to consider different experiences, backgrounds, and intersectional identities. For each finalized model, we computed false positive and false negative rates broken out by over 20 demographic categories, including intersectional identities. This allows us to ensure that we are correctly identifying risk levels and that no demographic group was favored or disfavored by the model.
THE RESULTS
For The Trevor Project, implementing AI models means being able to serve more youth, at a faster rate, with a consistently high quality of care.
Effective Automation: 100% of Trevor Digital cases are now automatically triaged through sophisticated natural language AI.
Improved prioritization: Asking automated pre-conversation questions allows Team Trevor to prioritize digital crisis counseling conversations with significantly more sensitivity and accuracy, and help high- and imminent-risk contacts get connected with our life-saving counselors as quickly as possible.
Reduced Bias: The model outperformed a baseline classifier that only used responses from several multiple-choice intake questions as features. The NLP model was also shown to have less bias than the baseline model.
Some of the highest-impact ingredients to the final model were:
Using in-domain unlabeled data to further pretrain an off-the-shelf ALBERT model
Encoding multiple-choice responses as full text, which is in turn encoded by ALBERT
Tuning hyperparameters using intuition about our specific dataset in addition to standard search methods.
Revamping our volunteer management system with the PwC Charitable Foundation
In November 2019, The PwC Charitable Foundation, Inc. awarded The Trevor Project $6 Million grant over four years — the largest grant we have ever received.
THE CHALLENGE
In order to meet the growing number of youth reaching out to Trevor, we needed to expand our volunteer base, but expansion proved difficult. Our tracking systems differed between our Volunteer, Training, and Crisis Services teams, creating challenges as volunteers progressed through stages of their journey led by different teams. This led to volunteers getting lost in the onboarding process. Time between interviewing, onboarding, and training also lagged, causing recruits to drop off.
We understood that to expand our volunteer base, we needed to revamp our technological infrastructure to effectively recruit and manage exponentially more volunteers.
THE SOLUTION
We partnered with PwC Charitable Foundation to deploy a new back-end volunteer management system that connects the following platforms: Okta, volunteer online application, Salesforce CRM, asynchronous training, and volunteer scheduling.
The team rapidly deployed technology capabilities and automation on the Salesforce platform — cutting development time by using pre-built components, out of the box functionality, and Custom Apex code, which ultimately allowed for more robust testing periods and made it possible to deliver a scalable product faster.
This streamlines several processes around recruitment, training, and retention, including:
Centralized data: For the first time, Trevor’s volunteer journey — from application to training to managing crisis counselors — is tracked in a single database that is accessible by all of our volunteer management teams.
Streamlined Registration and Automated User Provisioning: We implemented an Okta registration for a smoother user experience and a more simplified system provisioning process.
Revamping the Volunteer Application: We redesigned our volunteer application, improving the user experience and linking the application directly to Salesforce so applicant Contacts are automatically created to begin their volunteer journey.
Automating Emails: We implemented email tracking in Salesforce with Einstein Activity Capture and incorporated spam prevention and deliverability tracking via SendGrid integration (SMTP relay).
Streamlining Interview Scheduling: We embedded Calendly links with the recruiter’s updated schedule and integrated Calendly to update Interview Status in Salesforce to make scheduling easy and seamless.
We have also worked to expand our digital asynchronous training model — allowing accepted volunteer applicants to go through self-paced training from their home to become crisis counselors.
THE RESULTS
These enhancements have already significantly streamlined Trevor’s volunteer management, reducing cost per volunteer and allowing for seamless data-sharing and collaboration between teams. Volunteers engaged with The Trevor Project now receive streamlined communications and the onboarding process takes less time, allowing new volunteers to start helping at-risk youth more quickly.
Streamlining Volunteer Onboarding: Thanks to new digital efficiencies,we’re on track to increase crisis counselors tenfold, from about 300 to 3,000, by 2022.
Automating for Clear Communication: Our new system improves the applicant experience with automated emails when they apply and when they have been selected for an interview, ensuring that we have consistent, predictable touchpoints with our applicants.
Saving Valuable Time: automatedemail processes save volunteer recruiters approximately two hours a week. Centralizing data and consolidating volunteer information into one system has significantly decreased time spent working in spreadsheets and made internal communication more effective.
By May 2021, we will reimagine and deploy a new scheduling system, automating and streamlining scheduling to make it easier for volunteers to schedule shifts with flexibility for their schedules — a key aspect of managing and retaining 2,800 high-quality crisis counselors serving LGBTQ youth.
