COSS Community 🌱

Cover image for OCS 2020 Breakout: Mady Mantha, Rasa
Joseph (JJ) Jacks for COSS Community

Posted on

OCS 2020 Breakout: Mady Mantha, Rasa

Mady Mantha is a Senior Technical Evangelist at Rasa. Mady studied Computer Science, Physics, and International Politics. She has experience building ML-driven products and is a space enthusiast.

Relevant Links
LinkedIn - Twitter

In this talk, you’ll learn about how to improve conversational AI systems that work at scale in the real world.

Mady’s introduction and presentation topic: Build Enterprise-Grade AI assistants using Conversation-Driven Development - 0:00

Mady’s background - 0:17

Capture the lessons we’ve learned as a community in building conversational AI - 1:10

Rasa’s mission: Building the standard infrastructure for conversational AI - 2:58

Community showcase: - 3:32

The 5 levels of conversational AI (see: L3-AI keynote) - 4:28

5 levels of conversational AI from end-user perspective - 5:56

5 levels of conversational AI from developer perspective - 7:00

Walking through process/requirements for an AI assistant - 12:05

How you use Rasa within your project and how it fits into your stack - 13:37

Walking through Conversation-Driven Development (CDD) - 15:31

What is the alternative/ opposite of conversation-driven development? - 17:45

What each of the steps of CDD look like, starting with Share. (Users will always surprise you, test your prototype as early as possible. Find out how people will break out). - 19:16

Review (at every stage of project, read what users are saying. Avoid getting caught up in metrics right away). - 20:23

Annotate (improve NLU model based on messages from previous conversations. Keep less than 10% of data synthetic going into production.) - 21:00

Test (use whole conversations as end-to-end tests for the assistant.) - 21:32

Track (use proxy measures to track which conversations are successful and which ones failed, for example, whether a user clicked a link. ‘Negative’ signals are useful too, e.g. users not getting back in touch with support. Track all the way the assistants fail, end-to-end, so you can reduce failure over time.) - 22:02

Walking through Rasa X: a tool to help teams do CDD. Runs in browser, provides a UI to help you test and improve assistant. Rasa X is also free. - 23:26

What Rasa X looks like. First, share your bot with testers using just a link. Review conversations coming in from every channel (includes filtering, and creating new training data). Annotate the messages coming in as new NLU examples. Push new training data to git and trigger your CI pipeline. Track failures and successes (and turn successful conversations into new end-to-end tests). - 24:23

It’s a continuous iterative process, not a linear process. - 28:10

Some actions require software skills, others a deep understanding of the user. Requires a collaboration between development, DevOps, product, subject-matter experts, and the user, to make a great assistant that actually helps users. - 29:24

The community is at the heart of getting to the next level (see: - 30:14

Concluding remarks and contacts (Twitter: @madymantha, - 31:47

Share your questions and comments below!

Top comments (0)