Microsoft Teams is excellent collaboration tool with person to person communication workloads like, Messaging, Voice and Video collaboration. Microsoft Teams can also use Microsoft AI and cognitive services to collaborate with machines and devices. Together with the large suite of Azure services that allows me to call Azure apps to orchestrate person to machine control, remote diagnostics and telemetrics analytics of internet connected devices.
My Teams BOT is set up as a personal assistant that manages communications between several of my projects. The fact I can use a single interface to run many purchased and custom built apps displays the flexibility of Azure BOT’s. I currently run three personal custom created applications, a Office 365 management Assistant, a lockdown and alert system, and IOT device control, all through this single Teams BOT.
To demonstrate how Microsoft teams can control remote robotics, I have created a fun project that allows Teams to manage a RoboRaptor through Teams natural language messages.
The objective is to send control commands from MS Teams as natural language messages that are sent to a Microsoft AI BOT. The BOT will then use Azure LUIS language understanding services to determine the command intent. The result is sent to the Internet of Things controller card attached to the robo raptor for translation into machine commands.
The Robo Raptor and the MXCHIP is a working IOT device. Live telemetric data is sent back into Azure IOT HUB service to monitor environmental statistics which can be measured through Power BI. Temperature and humidity readings are typical of a standard IOT end point. The MXCHIP is configured with Arduino code which is very common microcontroller IDE platform.
The RoboRaptor project is complex and consumes multiple services from Azure. However, I have been able to build this solution with free tier services and so far I am up for $80 for the MXCHIP and dual relay module. The RoboRaptor was one of the kids old toys I saved from extinction.
The Robo Raptor Project uses the following Azure services.
The Project includes,
• Microsoft Teams for user interface
• BOTs for creating intelligent interaction and function calls to IOT and other Azure services
• Cognitive services, LUIS language understanding services to allow normal natural language, between user and robotics
• QNA, Question and Answer builder to create help menus and information repositories for users
• Facial Recognition Cognitive service, to scan people around the raptor and identify as owner or foe,
• Server-less Azure functions to control communications between IOT and Teams
• IOT, Azure internet of things services to manage and communicate with IOT hardware
• MXCHIP, A small microcontroller that I have attached to the raptor to provide secure internet communication to Azure IOT Hub. The MXCHIP will receive commands and send instructions to the Robo Raptor.
The Mxchip will activate power to the robotics and a fire a laser weapon through switched circuits. The MXCHIP also sends telemetry data back to AZURE for storage and analytics. Information include things like Temperature, Humidity, Pressure, Accelerometer, and Gyroscope info.
My choice of IOT hardware was the MX Chip. I found this development board easy to use and register to the Microsoft Azure IOT HUB. It is Arduino compatible and the board library easy to follow. I used a break out board to access IO pins to activate relays to turn on power and activate the laser. The hardware specs are as follows.
Device Summary:
Microcontroller: STM32F412RG ARM®32-bit Cortex®-M4 CPU
Operating Voltage: 3.3V
Input Voltage: 7-12V
Digital I/O Pins (DIO): 31
Analog Input Pins (ADC): 2
UARTs: 2
SPIs: 1
I2Cs: 1
Flash Memory: 1 MB
SRAM: 256 KB
Clock Speed: 100 MHz
The follow diagram shows the message flow between the MXChip and MS Teams.
Video footage in action https://youtu.be/E6lqlxsQsFw
The project blogs is broken up into the following key milestones
Microsoft Teams BOTS and cognitive services. Part 2 https://blog.kloud.com.au/2019/03/18/microsoft-teams-and-iot-controled-robotics-the-bot/
Microsoft IOT and the MXCHIP Part 3
Robo raptor Facial recognition Part 4