Posted by Mary Chen, Product Marketing Manager, and Ralfi Nahmias, Product Manager, Dialogflow
Today at Google Cloud Next '18, Dialogflow is introducing several new beta features to expand conversational capabilities for customer support and contact centers. Let's take a look at how three of these features can be used with the Google Assistant to improve the customer care experience for your Actions.
Building conversational Actions for content-heavy use cases, such as FAQ or knowledge base answers, is difficult. Such content is often dense and unstructured, making accurate intent modeling time-consuming and prone to error. Dialogflow's Knowledge Connectors feature simplifies the development process by understanding and automatically curating questions and responses from the content you provide. It can add thousands of extracted responses directly to your conversational Action built with Dialogflow, giving you more time for the fun parts – building rich and engaging user experiences.
Try out Knowledge Connectors in this bike shop sample
When users interact with the Google Assistant through text, it's common and natural to make spelling and grammar mistakes. When mistypes occur, Actions may not understand the user's intent, resulting in a poor followup experience. With Dialogflow's Automatic Spelling Correction, Actions built with Dialogflow can automatically correct spelling mistakes, which significantly improves intent and entity matching. Automatic Spelling Correction uses similar technology to what's used in Google Search and other Google products.
Enable Automatic Spelling Correction to improve intent and entity matching
Your Action can now be used as a virtual phone agent with Dialogflow's new Phone Gateway integration. Assign a working phone number to your Action built with Dialogflow, and it can start taking calls immediately. Phone Gateway allows you to easily implement virtual agents without needing to stitch together multiple services required for building phone applications.
Set up Phone Gateway in 3 easy steps
Dialogflow's Knowledge Connectors, Automatic Spelling Correction, and Phone Gateway are free for Standard Edition agents up to certain limits; for enterprise needs, see here for more options.
We look forward to the Actions you'll build with these new Dialogflow features. Give the features a try with the Cloud Next FAQ Action we made:
And if you're new to developing for the Google Assistant, join our Cloud Next talk this Thursday at 9am – see you on the livestream or in person!
The Google Assistant is becoming even more conversational and visual – helping people get things done, save time and be more present. And developers like you have been a big part of this story, making the Assistant more useful across more than 500 million devices. Starbucks, Disney, Zyrtec, Singapore Airlines and many others are engaging with users through the Actions they've built. In total, the Google Assistant is ready to help with over 1 million Actions, built by Google and all of you.
Ever since we launched Actions on Google, our mission has been to give you the tools you need to create engaging Actions, making them a part of people's everyday lives. Just over the past six months we've made significant upgrades to our platform to bring us closer to that vision. We made improvements to help your Actions get discovered, opened Actions on Google to more languages, took a few steps toward making your Actions more creative and visually appealing, launched a new conversation design site, and last week announced a new program to invest in startups that push the Assistant ecosystem forward.
Today, I want to share how we're making it even easier for app and web developers to get started with the Google Assistant.
We've seen a lot of great Android developers build Actions that complement their mobile apps. You can already create a personal, connected experience across your Android app and the Actions you build for the Assistant. Now we're making it possible to extend your Android app experiences to the Assistant in even more ways.
Think of your Actions for the Google Assistant as a companion experience to your app that users can access at home or on the go, across phones, smart speakers, TVs, cars, watches, headphones, and, soon, Smart Displays. If you want to personalize some of the experiences from your Android app, account linking lets your users have a consistent experience whether they're in your app or interacting with your Action.
We added support for seamless digital subscriptions so your users can enjoy the content and digital goods they bought in the Google Play Store right in your Assistant Action. For example, since I'm a premium subscriber in the Economist's app, I can now enjoy their premium content on any Assistant-enabled device.
And while you can already help users complete transactions for physical goods, soon you will be able to offer digital goods and subscriptions directly from your Actions.
The Assistant blends conversation with rich visual interactions for phones, Smart Displays and TVs. We've made it so your Actions already work on these visual surfaces with no extra work. Starting today, you can take this a step further and better customize the appearance of your Actions for visual surfaces by, among other things, controlling the background image, defining the typeface, and setting color themes used in your Action. Just head to the Actions console, make your changes and test them in the simulator today. These changes will be available on phones, TVs and Smart Displays, when they launch.
Here's an example screenshot from a demo Action:
And below, you can see how Volley was able to create a full screen immersive experience for their game "King for a Day." The ability to create customizable edge-to-edge visuals will launch for developers in the next few months.
In the Android keynote today, we announced a new feature called App Actions. App Actions are a new way to raise the visibility of your Android app to users as they start their tasks. We look forward to creating another channel to reach more users that can engage with your App Actions in the Google Assistant.
App Actions will be available for all developers to try soon; please sign up here if you'd like to be notified.
After you've built an Action for the Assistant, you want to get lots of people engaged with your experience. You can already prompt your users to sign up for Action Notifications on their phones, and soon, we'll be expanding support so users can get notifications on smart speakers and Smart Displays. Today we're also announcing three updates aimed at helping more users discover your Actions and keeping them engaged on a daily basis.
Map your Actions to users' queries with built-in intents
Over the past 20 years, Google has helped connect people with the information, services and content they're looking for by organizing, ranking, and showing the most relevant experience for users. With built-in intents, we're bringing this expertise to use in the Google Assistant. When someone says "Hey Google, let's play a maps quiz" they expect the Assistant to suggest relevant games that might pertain to geography. For that to happen, we need to understand the user's fundamental intent. This can be pretty difficult; just think of the thousands of ways a user could ask for a game.
To handle this complexity, we're beginning to map all the ways that people can ask for things into a taxonomy of built-in intents. Today, we're making the first set of these intents available to you so you can give the Assistant a deeper understanding of what your Action can do. As a result, the Assistant will be able to better understand and recommend Actions to meet a user's intent. We'll be rolling out hundreds of built-in intents in the coming months.
Today you can implement built-in intents in your action and test them in the simulator. You'll be able to use these in production soon.
Promote your Actions from anywhere a link works We're now making it easier to drive traffic to your Actions with Action Links. These are hyperlinks you can use anywhere—your website, emails, blog, even social media channels like Facebook and Twitter—that deep link directly into your Action. Now, when a developer like Headspace has something new to share, they can spread the word and drive engagement directly into their Action from across the web. Users can click on the link and jump into their Action's experience on phones and Smart Displays, and if they click the Action Link while on desktop, they can choose which Assistant-enabled device they'd like to use – from smart speakers to TVs. Go see an example on Headspace's website, or give their Action Link a try here.
If you've already built an Action and want to spread the word, starting today you can visit the Actions console to find your Action Links and get going.
Become a part of your users' daily routines
To consistently re-engage with users, you need to become a part of their daily habits. Google Assistant users can already use routines to execute multiple Actions with a single command, perfect for those times when users wake up in the morning, head out of the house, get ready for bed or many of the other tasks we perform throughout the day. Now, with Routine Suggestions, after someone engages with your Action, you can prompt them to add your Action to their routines with just a couple of taps.
So when I leave the house for work each morning, I can have my Assistant order my Americano from Starbucks and play that premium content from the Economist.
You can enable your Action for Routine Suggestions in the console today, and it will be working in production soon.
And more...
Before you run off and start sharing Actions links to all of your followers on social media, check out some of the other announcements we're making here at I/O:
Extend your experiences to the Google Assistant We're delighted to see that many of you are starting to test the waters in this emerging era of conversational computing. If you're already building mobile or web apps but haven't tried building conversational Actions for the Google Assistant just yet, now is the perfect time to get started. Start thinking of the companion experiences that could be a fit for the Google Assistant. We have easy-to-follow guides and a community program with rewards and Google Cloud credits to get you up and running in no time. We can't wait to try out your Actions soon!
Posted by April Pufahl, Conversation Designer
Creating Actions for the Google Assistant requires a breadth of design expertise ranging from voice user interface design, interaction design, visual design, motion design, and UX writing that we've refined into a single discipline: conversation design.
Today, we're launching a conversation design site that shares this expertise with you, so you can design Actions using the same principles that guide our teams at Google. Our goals are to help you:
If you're new to conversation design, you'll learn the basics of the conversation design process and how to determine whether conversation is right for your Action. You'll also get practical tips on how to:
Finally, we've broken down the conversational and visual components that are used to compose your Actions' responses to the user.
By following our conversation design principles, you'll adapt to the communication system users learned first and know best, and in the process, build better Actions.
Follow us on Twitter @ActionsOnGoogle and join our G+ community https://g.co/actionsdev to keep up to date with more news from our team.
Ever wanted to learn about developing for the Google Assistant and meet other developers that are passionate about conversational UI? Well, we've got some good news!
Today, we are launching a global series of events about Actions on Google, run by Google Developers Groups (GDG) and other community groups. In these events, you'll be able to meet other developers and go together through educational content, uniquely crafted for these events by Google engineers. This includes tutorials on how to build your first Action and advanced sessions on how to use more complex features of the platform. By the end of the event you attend, you'll be able to build an Action for your community - be it your hometown, your professional network, or interest group.
And if you don't see an event near you, don't worry - you can always organize your own. We'll help!
It's going to be a great year for Actions developers. Please join us and check out the dedicated event website with all the event details and more information: developers.google.com/events/buildactions!
Though it's been just a few short weeks since we released a new set of features for Actions on Google, we're kicking off our presence at South by Southwest (SXSW) with a few more updates for you.
SXSW brings together creatives interested in fusing marketing and technology together, and what better way to start the festival than with new features that enable you to be more creative, and to build new type of Actions that help your users get more things done.
This past year, we've heard from many developers who want to offer great media experiences as part of their Actions. While you can already make your podcasts discoverable to Assistant users, our new media response API allows you to develop deeper, more-engaging audio-focused conversational Actions that include, for example, clips from TV shows, interactive stories, meditation, relaxing sounds, and news briefings.
Your users can control this audio playback on voice-activated speakers like Google Home, Android phones, and more devices coming soon. On Android phones, they can even use the controls on their phone's notification area and lock screen.
Some developers who are already using our new media response API include The Daily Show, Calm, and CNBC.
To get started using our media response API, head over to our documentation to learn more.
And if your content is more visual than audio-based, we're also introducing a browse carousel for your Actions that allows you to show browsable content -- e.g., products, recipes, places -- with a visual experience that users can simply scroll through, left to right. See an example of how this would look to your users, below, then learn more about our browse carousel here.
While having a great user experience is important, we also want to ensure you have the right tools to re-engage your users so they keep coming back to the experience you've built. To that end, a few months ago, we introduced daily updates and push notifications as a developer preview.
Starting today, your users will have access to this feature. Esquire is already using it to send daily "wisdom tips", Forbes sends a quote of the day, and SpeedyBit sends daily updates of cryptocurrency prices to keep them in the know on market fluctuations.
As soon as you submit your Action for review with daily updates or push notifications enabled, and it's approved, your users will be able to opt into this re-engagement channel. Learn more in our docs.
Actions for Google now allows you to access digital purchases (including paid app purchases, in-app purchases, and in-app subscriptions) that your users make from your Android app. By doing so, you can recognize when you're interacting with a user who's paid for a premium experience on your Android app, and similarly serve that experience in your Action, across devices.
And the best part? This is all done behind the scenes, so the user doesn't need to take any additional steps, like signing in, for you to provide this experience. Economist Espresso, for example, now knows when a user has already paid for a subscription with Google Play, and then offers an upgraded experience to the same user through their Action.
In December of last year we announced the addition of Built-in Device Actions to the Google Assistant SDK for devices. This feature allows developers to extend any Google Assistant that is embedded in their device using traits and grammars that are maintained by Google and are largely focused on home automation. For example "turn on", "turn off" and "turn the temperature down".
Today we're announcing the addition of Custom Device Actions which are more flexible Device Actions, allowing developers to specify any grammar and command to be executed by their device. Once you build these Custom Device Actions, users will be able to activate specific capabilities through the Google Assistant. This leads to more natural ways in which users interact with their Assistant-enabled devices, including the ability to utilize more specific device capabilities.
Before:
"Ok Google, turn on the oven"
"Ok, turning on the oven"
After:
"Ok Google, set the oven to convection and preheat to 350 degrees"
"Ok, setting the oven to convection and preheating to 350 degrees"
To give you a sense of how this might work in the real world, check out a prototype, Talk to the Light from the talented Red Paper Heart team, that shows a zany use of this functionality. Then, check out our documentation to learn more about how you can start building these for your devices. We've provided a technical case study from Red Paper Heart and their code repository in case you want to see how they built it.
In addition to Custom Device Actions, we've also integrated device registration into the Actions on Google console, allowing developers to get up and running more quickly. To get started checkout the latest documentation and console.
Similarly, we teamed up with a few cutting-edge teams to explore the creative potential of the Actions on Google platform. Following the Voice experiments the Google Creative Lab released a few months ago, these teams released four new experiments:
The code for all of these Actions is open source and is accompanied by in-depth technical case studies from each team that shares their learnings when developing Actions.
Ready to build? Take a look at our three new case studies with KLM Royal Dutch Airlines, Domino's, and Ticketmaster. Learn about their development journey with Dialogflow and how the Actions they built help them stay ahead of the conversational technology curve, be where their customers are, and assist throughout the entire user journey:
We hope these updates get your creative juices flowing and inspire you to build even more Actions and embed the Google Assistant on more devices. Don't forget that once you publish your first Action you can join our community program* and receive your exclusive Google Assistant t-shirt and up to $200 of monthly Google Cloud credit. Thanks for being a part of our community, and as always, if you have ideas or requests that you'd like to share with our team, don't hesitate to join the conversation.
*Some countries are not eligible to participate in the developer community program, please review the terms and conditions
While Actions on the Google Assistant are available to users on more than 400 million devices, we're focused on expanding the availability of the developer platform even further. At Mobile World Congress, we're sharing some good news for our international developer community.
Starting today, you can build Actions for the Google Assistant in seven new languages:
These new additions join English, French, German, Japanese, Korean, Spanish, Brazilian Portuguese, Italian and Russian. That brings our total count of supported languages to 16! You can develop for all of them using Dialogflow and its natural language processing capabilities, or directly with the Actions SDK. And we're not stopping here–expect more languages to be added later this year.
If you localize your apps in these new languages you won't just be among the first Actions available in the new locales, you'll also earn rewards while you do it! And if you're new to Actions on Google, check out our community program* to learn how you can snag an exclusive Google Assistant t-shirt and up to $200 of monthly Google Cloud credit by publishing your first Action. Already we've seen partners take advantage of other languages we've launched in the past like Bring!, which is now available in both English and German.
Besides supporting new languages, we're also making it easier to build your Action for global audiences. First, we recently added support for building with templates—creating an Action by filling in a Google Sheet without a single line of code—for French, German, and Japanese. For example, TF1 built Téléfoot, using templates in French to create an engaging World Cup-themed trivia game with famous commentators included as sound effects.
Additionally, we've made it a little easier for you to localize your Actions into different languages by enabling you to export your directory listing information as a file. With the file in hand, you can translate offline and upload the translations to your console, making localization quicker and more organized.
But before you run off and start building Actions in new languages, take a quick tour of some of the useful developer features rolling out this week…
By the end of the year the Assistant will reach 95 percent of all eligible Android phones worldwide, and Actions are a great way for you to reach those users to help them get things done easily over voice. Sometimes, however, users may benefit from the versatility of your Android app for particularly complex or highly interactive tasks.
So today, we're introducing a new feature that lets you deep link from your Actions in the Google Assistant to a specific intent in your Android app. Here's an example of SpotHero linking from their Action to their Android app after a user purchased a parking reservation. The Android app allows the user to see more details about the reservation or redeem their spot.
As you integrate these links in your Action, you'll make it easier for your users to find what they're looking for and to move seamlessly to your Android app to complete their user journey. This new feature will roll out over the coming weeks, but you can check out our developer documentation for more information on how to get started.
We're also introducing askForPlace, a new conversation helper that integrates the Google Places API to enable developers to use the Google Assistant to understand location-based user queries mid-conversation. Using the new helper, the Assistant leverages Google Maps' location and points of interest (POI) expertise to provide fast, accurate places for all your users' location queries. Once the location details have been sorted out with the user, the Assistant returns the conversation back to your Action so the user can finish the interaction.
So whether your business specializes in delivering a beautiful bouquet of flowers or a piping hot pepperoni pizza, you no longer need to spend time designing models for gathering users' location requests, instead you can focus on your Action's core experience.
Let's take a look at an example of how Uber uses the askForPlace helper to help their users book a ride:
We joined halfway through the interaction above, but it's worth pointing out that once the Uber action asked the user "Where would you like to go?" the developer triggered the askForPlace helper to handle location disambiguation. The user is still speaking with Uber, but the Assistant handled all user inputs on the back end until a drop-off location was resolved. From there, Uber was able to wrap up the interaction and dispatch a driver.
Head over to the askForPlace docs to learn how to create a better user experience for your customers.
And to wrap up our new feature announcements, today we're introducing an improved experience for users who use your app regularly—without any work required on your end. Specifically, if users consistently come back to your app, we'll cut back on the introductory lead-in to get users into your Actions as quickly as possible.
Today's updates are part of our commitment to improving the platform for developers, and making the Google Assistant and Actions on Google more widely available around the globe. If you have ideas or requests that you'd like to share with our team, don't hesitate to join the conversation.
We recently launched a new YouTube video series focused on teaching developers best practices for the Actions on Google platform.
Apps for the Google Assistant are the gateway for users to engage with your services through Google Home, Android phones, iPhones, and in the future, through every experience where the Google Assistant is available.
The goal of the video series is to show you how to use the Google Assistant platform in the best way. You will learn more from Ido Green, Developer Advocate at Google, who will touch on topics like:
Tune in to learn how to build, or improve your apps for the Google Assistant so your users can benefit from more meaningful, interactive experiences.
And if you'd like to keep the conversation going, please join our developer community at: https://g.co/actionsdev or @actionsongoogle
See you!
Voice interaction has the potential to simplify the way we use technology. And with Dialogflow, Actions on Google, and Speech Synthesis API, it's becoming easier for any developer to create voice-based experiences. That's why we've created Voice Experiments, a site to showcase how developers are exploring voice interaction in all kinds of exciting new ways.
The site includes a few experiments that show how voice interaction can be used to explore music, gaming, storytelling, and more. MixLab makes it easier for anyone to create music, using simple voice commands. Mystery Animal puts a new spin on a classic game. And Story Speaker lets you create interactive, spoken stories by just writing in a Google Doc – no coding required.
You can try the experiments through the Google Assistant on your phone and on voice-activated speakers like the Google Home. Or you can try them on the web using a browser like Chrome.
It's still early days for voice interaction, and we're excited to see what you will make. Visit g.co/VoiceExperiments to play with the experiments or submit your own.
Last month we announced that UK users can access apps for the Google Assistant on Google Home and their phones—and starting today, we're bringing Actions on Google to Australia. From Perth to Sydney, developers can start building apps for the Google Assistant, giving their users even more ways to get things done.
Similar to our launch in the UK, your English apps will appear in the local directory automatically. With that said, there are a few things to help make your app a true blue Aussie:
Our developer tools, documentation and simulator have all been updated to make it easy for you to create, test and deploy your app. So what are you waiting for?
UK and Aussie users are just the start, we'll continue to make the Actions on Google platform available in more languages over the coming year. If you have questions about internationalization, please reach out to us on Stackoverflow and Google+.
Starting today, we're making all your apps built for the Google Assistant available to our en-GB users across Google Home (recently launched in the UK), select Android phones and the iPhone.
While your apps will appear in the local directory automatically this week, to make your apps truly local, here are a couple of things you should do:
Apps like Akinator, Blinkist Minute and SongPop have already optimized their experience for en-GB Assistant users—and we can't wait to see who dives in next!
And for those of you who are excited about the ability to target Google Assistant users on en-GB, now it is the perfect time to start building. Our developer tools, documentation and simulator have all been updated to make it easy for you to create, test and deploy your first app.
We'll continue to make the Actions on Google platform available in more languages over the coming year. If you have questions about internationalization, please reach out to us on Stackoverflow and Google+.
Cheerio!
