Posted by Aniedi Udo-Obong, Sub-Saharan Africa Regional Lead, Google Developer Communities
Samuel Mugisha has been very busy since we first shared his story in 2019. Back then, Samuel was a university student from Uganda who was inspired to create a mobile Immunization Calculator app to help keep track of children’s vaccinations after he saw his community using handwritten paper cards that were difficult to read and keep intact. Samuel kicked his idea into gear by forming a team with friends from GDSC Muni University, and they ended up winning the 2019 Google Developer Student Clubs Solution Challenge. The Solution Challenge is an annual event hosted by Google developers, inviting students from all over the world to develop solutions for local community problems using Google technology. Thanks to their win, Samuel and his team secured office space in Uganda and got up to speed with crucial tools like Android, Firebase, and Presto, with mentorship from some of Africa’s best at Google and other companies, as well as Google Developer Group community organizers.
Still, their project was in its earliest stages of development, and Samuel had a long way to go on the path to fixing a flawed healthcare system.
Fast forward to today, and Samuel and his team have made huge strides in building out the functionalities of their Immunization Calculator app. Lately, they’ve been working to increase the number of vaccinations the app can track and expanding to include other vaccines as well. These important updates will bring Samuel one step closer to his goal of increasing tracking and vaccinations of infants and children under a year old. Samuel and his team are also putting extra focus on improving functionality and user experience, making it easier than ever for parents to use the app to track their children’s vaccination status.
Vaccine tracking in-app
Thanks to the mentorship they received from Google Developer Experts as Solution Challenge winners, Samuel’s team was able to embed SMS messaging capabilities directly into the app. With this new functionality, the app now has the ability to remind parents to schedule their child’s next vaccine, and can provide accurate dosing information.
Vaccine reminder text alerts
In the app’s initial phase, the team relied on a simple user interface using NativeScript integrated to run on Gradle in Android Studio. Through mentorship, they found that implementing Kotlin instead was a more stable option that ran on most mobile devices. Since 2019, Felix Egaru, the core developer behind the project, has done a lot of work to add more features using Firebase. The app uses Firebase Authentication to authorize system users, Firebase Cloud Messaging to send push notifications, and Cloud Firestore to securely store user data. It also includes coverage and dropout rates, dosing instructions, and other important information thanks to Firebase. All of these new features let users access the data they need to make informed decisions about getting vaccinated. In a country with remote areas that have little to no internet access, Firebase’s offline capabilities have also proved vital for allowing healthcare workers to use the app in the field regardless of internet connectivity.
Despite all their progress, Samuel and his team still lacked the help they needed to get the app out into the world for real-time testing. Enter Kabagweri Fionah, a busy parent and small business owner who had started using Samuel’s app to keep track of her son’s vaccinations. She first discovered the Immunization Calculator app after seeing the Solution Challenge online and learning about the finalists.
Fionah immediately saw the app’s potential and knew that it could go a long way towards making a real difference in people’s lives, so she reached out to Samuel directly to see if she could join the team and help get the app out into the world. Not long after, Fionah was hired as a public relations officer, tasked with handling marketing, advocacy, and communications efforts to grow the app even further.
Kabagweri Fionah and Samuel Mugisha
Fionah joined the team at just the right time. Through her outreach efforts, Samuel and his team gained access to a local hospital in Kampala where they could actually put the Immunization Calculator to the test. Beyond just spreading the word, testing at the hospital helped them identify and fix bugs in the system and prepare their app for future, larger-scale testing. Recently, Samuel and his team reached an even bigger milestone. Thanks to Fionah’s persistence, the team was given an opportunity to deploy their project at the biggest health center in Kyegegwa District. Samuel and his team are currently working to raise funds so that they can complete a full year of system testing at the center.
Samuel at the health center in Kyegegwa District
As Samuel explains, Google Developer Student Clubs has helped his team get their foot in the door all along the way:
Samuel credits the Solution Challenge and Google Developer Student Clubs for helping him transform his brilliant idea into a life-changing application. He puts it best when explaining why winning the Solution Challenge was such a huge motivation for him and his team.
Thanks to Google Developer Student Clubs, Samuel found a team of bold thinkers with the same passion to help others, and the drive to turn an idea into reality. Google Developer Experts were crucial in helping them bring the app’s functionalities to the next level. Samuel and his peers were given access to the tools they needed to grow: Coursera, Qwiklabs, Pluralsight, and Google Cloud credits to polish their development skills.
This year’s Solution Challenge Demo Day is right around the corner on August 26th, where the top 10 finalists will showcase their projects to Google and developers around the world live on YouTube for a chance to be named one of the final 3 winning teams. Samuel knows what the pressure feels like, and he’s got some great advice for the current finalists:
Thanks to Samuel’s focus on the goal and drive to make a difference, he and his team are well on their way to making healthcare more accessible for Ugandans in his community and beyond. Google Developer Student Clubs are perfect for building solutions to help your community with other students who share your passion. Equip yourself with the knowledge you need to bring your best idea to life. Click here to find a Google Developer Community Student Club community near you.
Posted by Kubra Zengin, North America Regional Lead, Google Developer Groups
When asked to speak to a room full of incarcerated individuals about becoming developers, Danny Thompson didn’t bat an eye. Danny is an experienced software engineer and community organizer for Google Developer Groups Memphis.
But for the first ten years of his professional career, he worked in a gas station frying chicken. If anyone knows how to beat the odds and choose a different path in life, it’s him.
(Left) Danny Thompson working in a gas station (Right) At work in the tech industry
Danny Thompson is a big believer in the power of community. Over the years, he’s grown a vast network of thousands of aspiring developers, tech industry professionals, and career development experts through the Google Developer Group community and across social media. So it was no surprise when Danny was contacted to speak at an event hosted by Persevere, a non-profit that teaches justice-involved individuals how to code and helps them find careers as developers. By teaching skills like programming, Persevere has seen a decrease in recidivism rates. Through their job placement efforts, they are helping those that get out, stay out.
For Danny, signing up to help out was an easy decision. His biggest motivator in life is helping others succeed, no matter their hardships or where they come from.
In March of last year, Danny spoke to a room full of incarcerated people at the non-profit’s secure facility in Memphis, Tennessee. Incarcerated individuals from Tennessee prisons are brought to this offsite location for hands-on training with coding instructors. For hours, Danny taught Javascript concepts, gave career guidance, and shared his best tips for growing in the tech industry with program participants.
Thanks to Danny’s role as community organizer for GDG Memphis, he was able to share many learning materials that came directly from Google Developer Group events. Those enrolled in the program were also given access to Google Developer Group events online to help them learn new concepts and network with other developers.
Danny recognizes how difficult it can be for incarcerated individuals to reenter society without a helping hand and the skills they need to attain a well-paying job. As a result, many previously incarcerated people return to prison because they are unable to find employment and have higher chances of falling back into bad habits. But when they learn to code, recidivism rates drop dramatically.
Program participants continue learning at a transition center in Memphis, Tennessee
Danny knows what it’s like to not fit the mold of a typical developer. After working ten years as a fry cook at a gas station, he never considered that transitioning to a career in technology was even possible. However, everything changed for Danny when he began attending Google Developer Group meetups.
By networking with other developers, Danny gained the skills he needed to grow his early interest in coding into actual opportunity. Fast forward to today, and Danny is using his connection to Google Developer Groups to break barriers for anyone and everyone interested in pursuing a career in the tech industry. As an organizer for GDG Memphis, Danny designs mentorship opportunities with experts in the tech industry and hosts meetups that connect aspiring developers to hiring managers. Through opportunities just like these, Danny has helped over 600 people land jobs in tech, and he’s not stopping anytime soon.
GDG Memphis meetup event
It’s never too late to join your local Google Developer Group. Learn new skills, advance your career, and meet other developers who share your interests. Anyone interested in tech is welcome, and joining is completely free.
To find your local chapter, click here.
Posted by Jason Scott, Head of Startup Developer Ecosystem, USA
Last year, 12 inspiring entrepreneurs kicked off the inaugural Google for Startups Accelerator for Black Founders. Throughout the three month program, founders met weekly to work on growing their startups and solving tough technical challenges. “There’s so much happening every single day as a startup,” says Ashley Edwards, founder of MindRight Health, whose startup is making mental healthcare accessible to people of color and low-income families. “The program helped us navigate everything from protecting our team from distractions to building out our machine learning and data science models.”
This August, we’ll launch the second Google for Startups Accelerator for Black Founders with 11 more incredible Black-led startups from across North America. This class features startups using technology to solve challenges in medicine, education, water sustainability, real estate, and more:
Acclinate (Birmingham, Alabama, USA): A digital health startup using culture and tech to source diverse participants for clinical trials.
Adapdix (Pleasanton, California, USA): An AI/ML startup that works with large industrial semiconductor, electronic and assembly companies.
AllHere Education (Boston, Massachusetts, USA): Fosters student attendance and supports families and students with mobile messaging powered by AI.
Chatdesk (New York, New York, USA): Uses machine learning to scale support teams with the click of a button.
DOSS (Houston, Texas, USA): A digital, voice-activated real estate marketplace that empowers consumers to speak, text or type questions about properties nationwide and receive accurate, easy answers instantly.
Fêtefully (Dallas, Texas, USA): Digitizes wedding planning experiences, allowing planners to generate greater revenue and improve their offerings to customers.
Mommy Monitor (Toronto Ontario, Canada): A maternal care services platform that provides an easily accessible and culturally safe range of services that gives parents extra support customized to their particular needs and wants.
Optimal Technical Corporation (Atlanta, Georgia, USA): Intelligently eliminates electricity waste, lowers operational expenses, and helps to save the planet.
Sugar (Los Angeles, California, USA): Provides software to building owners and managers to transform the residential experience.
Varuna (Chicago, Illinois, USA): The leading water distribution system monitoring company providing real-time visibility, awareness and insights to water systems enabling optimal operations and consumer safety.
Zirtue (Dallas, Texas, USA): The world's first relationship-based lending application that simplifies loans between friends, family and trusted relationships while giving borrowers the option to pay creditors directly using their borrowed funds.
We are incredibly excited to support this group of founders over the next three months and beyond, connecting them with the best of our people, products, and programming to advance their companies and solutions.
Be sure to join us as we showcase their accomplishments on Thursday, October 21 from 12:30pm - 2:00pm EST at our Google for Startups Accelerator: Black Founders Demo Day 2021.
Posted by Wesley Chun (@wescpy), Developer Advocate, Google Cloud
Today we're introducing the first video showing long-time App Engine developers how to migrate from the App Engine ndb client library that connects to Datastore. While the legacy App Engine ndb service is still available for Datastore access, new features and continuing innovation are going into Cloud Datastore, so we recommend Python 2 users switch to standalone product client libraries like Cloud NDB.
ndb
This video and its corresponding codelab show developers how to migrate the sample app introduced in a previous video and gives them hands-on experience performing the migration on a simple app before tackling their own applications. In the immediately preceding "migration module" video, we transitioned that app from App Engine's original webapp2 framework to Flask, a popular framework in the Python community. Today's Module 2 content picks up where that Module 1 leaves off, migrating Datastore access from App Engine ndb to Cloud NDB.
webapp2
Migrating to Cloud NDB opens the doors to other modernizations, such as moving to other standalone services that succeed the original App Engine legacy services, (finally) porting to Python 3, breaking up large apps into microservices for Cloud Functions, or containerizing App Engine apps for Cloud Run.
App Engine's Datastore matured to becoming its own standalone product in 2013, Cloud Datastore. Cloud NDB is the replacement client library designed for App Engine ndb users to preserve much of their existing code and user experience. Cloud NDB is available in both Python 2 and 3, meaning it can help expedite a Python 3 upgrade to the second generation App Engine platform. Furthermore, Cloud NDB gives non-App Engine apps access to Cloud Datastore.
As you can see from the screenshot below, one key difference between both libraries is that Cloud NDB provides a context manager, meaning you would use the Python with statement in a similar way as opening files but for Datastore access. However, aside from moving code inside with blocks, no other changes are required of the original App Engine ndb app code that accesses Datastore. Of course your "YMMV" (your mileage may vary) depending on the complexity of your code, but the goal of the team is to provide as seamless of a transition as possible as well as to preserve "ndb"-style access.
with
The "diffs" between the App Engine ndb and Cloud NDB versions of the sample app
To try this migration yourself, hit up the corresponding codelab and use the video for guidance. This Module 2 migration sample "STARTs" with the Module 1 code completed in the previous codelab (and video). Users can use their solution or grab ours in the Module 1 repo folder. The goal is to arrive at the end with an identical, working app that operates just like the Module 1 app but uses a completely different Datastore client library. You can find this "FINISH" code sample in the Module 2a folder. If something goes wrong during your migration, you can always rollback to START, or compare your solution with our FINISH. Bonus content migrating to Python 3 App Engine can also be found in the video and codelab, resulting in a second FINISH, the Module 2b folder.
All of these learning modules, corresponding videos (when published), codelab tutorials, START and FINISH code, etc., can be found in the migration repo. We hope to also one day cover other legacy runtimes like Java 8 and others, so stay tuned! Developers should also check out the official Cloud NDB migration guide which provides more migration details, including key differences between both client libraries.
Ahead in Module 3, we will continue the Cloud NDB discussion and present our first optional migration, helping users move from Cloud NDB to the native Cloud Datastore client library. If you can't wait, try out its codelab found in the table at the repo above. Migrations aren't always easy; we hope this content helps you modernize your apps and shows we're focused on helping existing users as much as new ones.
#IamaGDE series presents: Google Maps
Welcome to #IamaGDE - a series of spotlights presenting Google Developer Experts (GDEs) from across the globe. Discover their stories, passions, and highlights of their community work.
Homing Tam is a product manager at Lalamove, an on-demand logistics company. He started at the company as a product manager focusing on location-based systems, talking with developers and business users to enhance the company’s mapping solutions, before moving into product management. Now, Homing handles corporate solutions; takes care of people who want to integrate with his company’s systems; handles the API side of things to help make integration easier; and provides recommendations for developers and other technical teammates.
Homing studied geomatics and computing at university, and his 2009 thesis was based on Google’s API backend. His dissertation focused on using the Google Maps API to perform mapping and overlay. His first full-time job was as a GIS analyst at Esri, the largest private software company in the world. A year and a half later, he became a solutions consultant for a different company, helping customers interested in integrating Google Maps with their software.
After Homing got involved in the Google Technology User Group (now known as Google Developer Groups), his boss at the time told him about the Google Developer Experts program. For his interview, Homing presented a product using the Google Maps APIs. When he became a GDE, he gave presentations and talks in the greater China region as a surrogate for the Google Maps Platform team. Homing is currently one of the organizers for GDG Hong Kong, organizing and giving community talks.
Homing says the Maps Styling Wizard, the precursor to the newer Cloud-based Maps Styling features, is one of his favorite features.
“Cartography, which I studied in college, matters a lot, especially to a simple black and white schematic map, or when matching the theme of a map to a site,” he says. “I like that feature a lot.”
In 2020, Homing gave one talk on Android in the Android 11 Meetup and another talk on Maps at the first-ever virtual Hong Kong Devfest, and he’s ready to do more speaking.
“It had been a while since I gave a talk on maps, and the launch of Cloud-based Maps Styling is so exciting that I feel like it’s time to do some presentations and let the community know more about it. Beyond knowing how to use the API, you need to know how you can make the most of the API.”
Homing notes that this year, in particular, more small business owners need to know how to collect customer addresses, allow customers to place on-demand delivery orders, and update customers.
In 2021, in addition to giving more talks, Homing hopes to work with the GDG organizers in Hong Kong to plan a hackathon or otherwise teach community members more about the new Maps features.
“Can we make an MVP or a really initial stage cycling app to use as a base to explore the new features and use different Google components?’
As his career continues, Homing says he has two priorities: progressing as a product manager and leveraging technology, including maps, to improve lives.
“This year was a year for everyone to become digitally literate,” he says. “With the extra time we spend on technology, we should make good use of technology to make life better.”
For more information on Google Maps Platform, visit our website.
For more information on Google Developer Experts, visit our website.
Posted by Erica Hanson, Global Program Manager, Google Developer Student Clubs
Image of Olly and Daniel from Google Developer Student Clubs at Wash U.
When Olly Cohen first arrived on campus at Washington University in St. Louis (Wash U), he knew the school was home to many talented and eager developers, just like him. Computer science is one of the most popular majors at Wash U, and graduates often find jobs in the tech industry. With that in mind, Olly was eager to build a community of peers who wanted to take theories learned in the classroom and put them to the test with tangible, real-life projects. So he decided to start his own Google Developer Student Club, a university-based community group for students interested in learning about Google developer technology.
Olly applied to become Google Developer Student Club Lead so he could start his own club with a faculty advisor, host workshops on developer products and platforms, and build projects that would give back to their community.
He didn’t know it at the time, but starting the club would eventually lead him to the most impactful development project of his early career — building a web application with the potential to help front-line healthcare workers in St. Louis, Missouri, during the pandemic.
The Google Developer Student Club grew quickly. Within the first few months, Olly and the core team signed up 150 members, hosted events with 40 to 60 attendees on average and began working on five different projects. One of the club’s first successful projects, led by Tom Janoski, was building a tool for the visually impaired. The app provides audio translations of visual media like newspapers and sports games.
This success inspired them to focus their projects on social good missions, and in particular helping small businesses in St. Louis. With a clear goal established, the club began to take off, growing to over 250 members managed by 9 core team members. They were soon building 10 different community-focused projects, and attracting the attention of many local leaders, including university officials, professors and organizers.
As the St. Louis community began to respond to the coronavirus pandemic in early 2020, some of the leaders at Wash U wondered if there was a way to digitally track PPE needs from front-line health care staff at Wash U’s medical center. The Dean of McKelvey School of Engineering reached out to Olly Cohen and his friend Daniel Sosebee to see if the Google Developer Student Club could lend a hand.
The request was sweeping: Build a web application that could potentially work for the clinical staff of Wash U’s academic hospital, Barnes-Jewish Hospital.
So the students got right to work, consulting with Google employees, Wash U computer science professors, an industry software engineer, and an M.D./Ph.D. candidate at the university’s School of Medicine.
With the team assembled, the student developers first created a platform where they could base their solution. Next, they built a simple prototype with a Google Form that linked to Google Sheets, so they could launch a pilot. Lastly, in conjunction with the Google Form, they developed a serverless web application with a form and data portal that could let all staff members easily request new PPE supplies.
In other words, their solution was showing the potential to help medical personnel track PPE shortages in real time digitally, making it easier and faster to identify and gather the resources doctors need right away. A web app built by students poised to make a true difference, now that is what the Google Developer Student Club experience is all about.
Are you a student who also wants to use technology to make a difference in your community? Click here to learn more about joining or starting a Google Developer Student Club near you.
Posted by Charles Maxson, Developer Advocate
The Google Calendar API has changed how it manages API usage. Previously, queries were monitored and limited on a daily basis. As of May 2021, queries started to be monitored and limited on a per-minute basis. This introduces better behavior when your quota is exceeded, as requests are rate-limited until quota is available rather than failing all requests for the rest of the day. This also helps developers recognize issues around quota enforcements faster and shouldn't affect the performance of existing projects.
To view your usage and quota limits, have a look in the Google API console.
To help you manage your quotas, we’ve put together a few helpful tips:
For further details into managing quotas please review the manage quotas documentation. You can also find more details on error handling on the resolve errors documentation.
To stay on top of news and updates around Google Workspace APIs and developer platform please sign up to our developer newsletter.
The Google Cloud team recently introduced a series of codelabs (free, self-paced, hands-on tutorials) and corresponding videos designed to help users on one of our serverless compute platforms modernize their apps, with an initial focus on our earliest users running their apps on Google App Engine. We kick off this content by showing users how to migrate from App Engine's webapp2 web framework to Flask, a popular framework in the Python community.
While users have always been able to use other frameworks with App Engine, webapp2 comes bundled with App Engine, making it the default choice for many developers. One new requirement in App Engine's next generation platform (which launched in 2018) is that web frameworks must do their own routing, which unfortunately, means that webapp2 is no longer supported, so here we are. The good news is that as a result, modern App Engine is more flexible, lets users to develop in a more idiomatic fashion, and makes their apps more portable.
For example, while webapp2 apps can run on App Engine, Flask apps can run on App Engine, your servers, your data centers, or even on other clouds! Furthermore, Flask has more users, more published resources, and is better supported. If Flask isn't right for you, you can select from other WSGI-compliant frameworks such as Django, Pyramid, and others.
In this "Module 1" episode of Serverless Migration Station (part of the Serverless Expeditions series) Google engineer Martin Omander and I explore this migration and walk developers through it step-by-step.
In the previous video, we introduced developers to the baseline Python 2 App Engine NDB webapp2 sample app that we're taking through each of the migrations. In the video above, users see that the majority of the changes are in the main application handler, MainHandler:
MainHandler
Upon (re)deploying the app, users should see no visible changes to the output from the original version:
Today's video picks up from where we left off: the Python 2 baseline app in its Module 0 repo folder. We call this the "START". By the time the migration has completed, the resulting source code, called "FINISH", can be found in the Module 1 repo folder. If you mess up partway through, you can rewind back to the START, or compare your solution with ours, FINISH. We also hope to one day provide a Python 3 version as well as cover other legacy runtimes like Java 8, PHP 5, and Go 1.11 and earlier, so stay tuned!
All of the migration learning modules, corresponding videos (when published), codelab tutorials, START and FINISH code, etc., can all be found in the migration repo. The next video (Module 2) will cover migrating from App Engine's ndb library for Datastore to Cloud NDB. We hope you find all these resources helpful in your quest to modernize your serverless apps!
Written by: Jessica Dene Earley-Cha, Mike Bifulco and Toni Klopfenstein, Developer Relations Engineers for Google Assistant
Now that we’ve packed up all of the virtual stages from Google I/O 2021, let's take a look at some of the highlights and new product announcements for App Actions, Conversational Actions, and Smart Home Actions. We also held a number of amazing live events and meetups that happened during I/O - which we’ll summarize as well.
App Actions allows developers to extend their Android App to Google Assistant. For our Android Developers, we are happy to announce that App Actions is now part of the Android framework. With the introduction of the beta shortcuts.xml configuration resource and our latest Google Assistant Plug App Actions is moving closer to the Android platform.
Capabilities is a new Android framework API that allows you to declare the types of actions users can take to launch your app and jump directly to performing a specific task. Assistant provides the first available concrete implementation of the capabilities API. You can utilize capabilities by creating shortcuts.xml resources and defining your capabilities. Capabilities specify two things: how it's triggered and what to do when it's triggered. To add a capability, use Built-In intents (BIIs), which are pre-built intents that provide all the Natural Language Understanding to map the user's input to individual fields. When a BII is matched by the user’s speech, your capability will trigger an Android Intent that delivers the understood BII fields to your app, so you can determine what to show in response.
This framework integration is in the Beta release stage, and will eventually replace the original implementation of App Actions that uses actions.xml. If your app provides both the new shortcuts.xml and old actions.xml, the latter will be disregarded.
Google Assistant suggests relevant shortcuts to users and has made it easier for users to discover and add shortcuts by saying “Hey Google, shortcuts.”
You can use the Google Shortcuts Integration library, currently in beta, to push an unlimited number of dynamic shortcuts to Google to make your shortcuts visible to users as voice shortcuts. Assistant can suggest relevant shortcuts to users to help make it more convenient for the user to interact with your Android app.
Not only can Assistant suggest shortcuts, with In-App Promo SDK you can proactively suggest shortcuts in your app for actions that the user can repeat with a voice command to Assistant, in beta. The SDK allows you to check if the shortcut you want to suggest already exists for that user and prompt the user to create the suggested shortcut.
To support testing Capabilities, Google Assistant plugin for Android Studio was launched. It contains an updated App Action Test Tool that creates a preview of your App Action, so you can test an integration before publishing it to the Play store.
Learn more with new or updated content:
During the What's New in Google Assistant keynote, Director of Product for the Google Assistant Developer Platform Rebecca Nathenson mentioned several coming updates and changes for Conversational Actions.
Over the coming weeks, we’ll introduce new functionality to Interactive Canvas. Canvas developers will be able to manage intent fulfillment client-side, removing the need for intermediary webhooks in some cases. For use cases which require server-side fulfillment, like transactions and account linking, developers will be able to opt-in to server-side fulfillment as needed.We’re also introducing a new function, outputTts(), which allows you to trigger Text to Speech client-side. This should help reduce latency for end users.
outputTts()
Additionally, there will be updates to the APIs available to get and set storage for both the home and individual users, allowing for client-side storage of user information. You’ll be able to persist user information within your web app, which was previously only available for access by webhook.
These new features for Interactive Canvas will be made available soon as part of a developer preview for Conversational Actions Developers. For more details on these new features, check out the preview page.
Also coming soon to Conversational Actions - we’re updating the workflow for completing transactions, allowing users to complete transactions from their smart screens, by confirming the CVC code from their chosen payment method. Watch our demo video showing new transaction features on smart devices to get a feel for these changes.
Make sure to catch our technical session Driving a successful launch for Conversational Actions to learn about some strategies for putting together a marketing team and go-to-market plan for releasing your Conversational Action.
If you’re interested in building Games for Google Assistant with Conversational Actions, you should check out the recording of our AMA, where Googlers answered questions from I/O attendees about designing, building, and launching games.
The What's new in Smart Home keynote covered several updates for Smart Home Actions. Following our continued emphasis on quality smart home integrations with the updated policy launch, we added new features to help you build engaging, reliable Actions for your users.
The updated Test Suite for Smart Home now supports automatic testing, without the use of TTS. Additionally, the Analytics dashboards have been expanded with more detailed logs and in-depth error reporting to help you more quickly identify any potential issues with your Action. For a deeper dive into these enhancements, try out the Debugging the Smart Home workshop. There are also two new debugging codelabs to help you get more familiar with using these tools to improve the quality of your Action.
We expanded support for proactive notifications to include the device traits RunCycle and SensorState. Users can now be proactively notified for multiple different device events. We also announced the release of follow-up responses. These follow-up responses enable your smart devices to notify users asynchronously to device changes succeeding or failing.
We added support for WebRTC to the CameraStream trait. Smart camera users can now benefit from lower latency and half-duplex talk between devices. As mentioned in the keynote, we will also be making updates to the other currently supported protocols for smart cameras.
To improve the on-boarding experience, developers can now enable BLE (bluetooth low energy) for device onboarding with Bluetooth Seamless Setup. Google Home and Nest devices can act as local hubs to provision and register nearby devices for any Action configured with local fulfillment.
Project CHIP has officially rebranded as Matter. Once the IP-based connectivity protocol officially launches, we will be supporting devices running the protocol. Watch the Getting started with Project CHIP tech session to learn more.
Voice AI is fundamentally changing how we interact with technology and its future will be a product of the people that build it. Watch this session to hear about the talented women shaping the Voice AI field, including an interview with Lilian Rincon, Sr. Director of Product Management at Google. Leslie also discusses strategies for achieving equal gender representation in Voice AI, an ambitious but essential goal.
This "Ask Me Anything" session was hosted by the all-star team who runs the Google for Startups Accelerator: Voice AI. The team fielded questions from startups and investors around the world who are interested in building businesses based on voice technology. Check out the recording of this event here. The day after the AMA session, the 2021 cohort for the Voice AI accelerator had their demo day - you can catch the recording of their presentations here.
We connected with amazing women in Voice AI and discussed ways allies can help women in Voice to be more successful while building a more inclusive ecosystem. It was hosted by Leslie Garcia-Amaya, Jessica Dene Earley-Cha, Karina Alarcon, Mike Bifulco, Cathy Pearl, Toni Klopfenstein, Shikha Kapoor & Walquiria Saad
One of the perks of I/O being virtual this year was the ability to connect with students, hobbyists, and developers around the globe to discuss the current state of Smart Home, as well as some of the upcoming features. We hosted 3 meetups for the APAC, Americas, and EMEA regions and gathered some great feedback from the community.
Every year we host an Assistant Google Developer Expert meetup to connect and share knowledge. This year we were able to invite everyone who is interested in building for Google Assistant to network and connect with one another. At the end several attendees came together at the Assistant Sandbox for a virtual photo!
Thanks for reading! To share your thoughts or questions, join us on Reddit at r/GoogleAssistantDev.
Follow @ActionsOnGoogle on Twitter for more of our team's updates, and tweet using #AoGDevs to share what you’re working on. Can’t wait to see what you build!
Posted by Richard Adem, UX Engineer at Google Arts & Culture
One of the best ways to learn about global culture is by trying on famous art pieces using Google’s Augmented Reality technology on your mobile device. What does it feel like to wear a three thousand year old necklace, put on a sixteenth century Japanese helmet or don pearl earrings and pose in a Vermeer?
Google Arts & Culture have created a new feature called Art Filter allowing everyone to learn about culturally significant art pieces from around the world and put themselves inside famous paintings, normally safely displayed in a museum.
We teamed up with the MediaPipe team, which offers cross-platform, customizable ML solutions to combine ML with rendering to generate stunning visuals.
Working closely with the MediaPipe team to utilize their face mesh and 3D face transform allowed us to create custom effects for each of the artifacts we had chosen, and easily display them on as part of the Google Arts & Culture iOS and Android app.
Figure 1. The Art Filter feature.
We selected five iconic cultural treasures from around the world:
Given their diverse formats or textures each artwork or object required special approaches to bring it to life in AR.
Figure 2. User wearing the jewelry from Johannes Vermeer's "Girl with a Pearl Earring" - Mauritshuis museum, Hague.
Some of the artwork we selected are 2D paintings and we wanted everyone to immerse themselves in the paintings. Our team of 3D artists and designers took high resolution gigapixel images from Google Arts & Culture and projected them onto 3D meshes to texture them. We also extended the 2D textures all the way around the 3D meshes while maintaining the style of the original artist. This means that when you turn your head the previously hidden parts of the piece are viewable from every angle, mimicking how the object would look in real-life.
Figure 3. The Van Gogh Self-Portrait filter - Musée d’Orsay, Paris.
Our cultural partners were immensely helpful during the creation of Art Filter. They have sourced a huge amount of reference images allowing us to reproduce the pieces accurately using photographs from different angles, to help them appear to fit into the “real world” in AR (using size comparisons).
Art Filter takes an image of the user from their device’s camera and uses that to generate a 3D mesh of the user’s face. All processing of user images or video feeds is run entirely on device. We do not use this feature to identify or collect any personal biometric data; the feature cannot be used to identify an individual.
The image is then reused to texture the face mesh, generated in real-time on-device with MediaPipe Face Mesh, representing it in the virtual 3D world within the device. We then add virtual 2D and 3D layers around the face to complete the effect. The Tengu Helmet, for example, sits on top of the face mesh in 3D and is “attached” to the face mesh so it moves around when the user moves their head around. The Vermeer earrings with a headscarf and Frida Kahlo’s necklace are attached to the user’s image in a similar way. The Van Gogh effect works slightly differently since we still use a mesh of the user’s face but this time we apply a texture from the painting.
We use 2D elements to complete the scene as well, such as the backgrounds in the Kahlo and Van Gogh paintings. These were created by carefully separating the painting subjects from the background then placing them behind the user in 3D. You may notice that Van Gogh’s body is also 2D, shown as a “billboard” so that it always faces the camera.
Figure 4. Creating the 3D mesh showing layers and masks.
To create a realistic looking material we used “Physically Based” Rendering shaders. You can see this on the Tengu helmet, it has a bumpy surface that is affected by the real life light captured by the device. This requires creating extra textures, texture maps, for the effect that uses colors to represent how bumpy or shiny the 3D object should appear. Texture maps look like bright pink and blue images but tell the renderer about tiny details on the surface of the object without creating any extra polygons, which can slow down the frame rate of the feature.
Figure 5. User wearing Helmet with Tengu Mask and Crows - The Metropolitan Museum of Art.
We hope you enjoy the collection we have created in Art Filter. Please visit and try for yourself! You can also explore more amazing ML features with Google Arts & Culture such as Art Selfie and Art Transfer.
We hope to bring many more filters to the feature and are looking forward to new features from MediaPipe.
