3rd of March, 2017 / / No Comments

In the previous post, we built an ASP.NET Core application using Vue.js and TypeScript. As a working example, we’re building a mobile web application. Many modern web browsers supporting HTML5 can access to multimedia devices on users’ computer, smartphones or tablets, such as camera and microphone. The Navigator.getUserMedia() API enables us to access to those resources. In this post, we’re actually going to implement a feature for camera access on our computer and mobile devices, by writing codes in VueJs and TypeScript.

The code samples used for this post can be found here.

getUserMedia() API

Most modern web browsers support this getUserMedia() API, as long as they support HTML5. There are two different APIs around this method – one is Navigator.getUserMedia() that supports callback functions, while the other MediaDevices.getUserMedia(), that came up later, supports Promise so that we can avoid Callback Hell. However, not all browsers support the MediaDevices.getUserMedia(), so we need to support both anyway. For more details around getUserMedia(), we can find some practical samples in this MDN document.

Prerequisites

  • ASP.NET Core application from the previous post
  • Computer, tablet or smartphone having camera

NOTE 1: This post uses VueJs 2.2.1 and TypeScript 2.2.1. VueJs 2.2.1 introduced some breaking changes how it interacts with TypeScript. Please have a look at the official guide document.
NOTE 2: vue-webcam written by @smronju was referenced for camera access, and modified to fit in the TypeScript format.

Update Hello.vue

We need a placeholder for camera access and video streaming. Add the following HTML codes into the template section in Hello.vue.

  • video accepts the camera input. src, width, height and autoplay are bound with the component in Hello.ts. Additionally, we add the ref attribute for the component to recognise the video tag.
  • img is where the camera input is rendered. The photo field is used for data binding.
  • button raises the mouse click or finger tab event by invoking the takePhoto function.

The HTML bits are done. Let’s move on for TypeScript part.

Update Hello.ts

The existing Hello.ts was simple, while this time it’s grown up to handle the camera API. Here’s the bits:

We can see many extra data fields for two-way data binding between user input and application. Some of them comes with their default values so that we don’t have to worry about their initialisation too much.

  • The takePhoto() function creates a virtual DOM for canvas, converts the input signal from the video into an image, and sends it to the img tag to display snapshot.
  • The mounted() event function is invoked when this component, Hello.ts, is mounted to its parent. It uses the getUserMedia() API to bind streaming source to the video tag.
  • The video tag through this.$refs.video is the HTML element that has the ref attribute in Hello.vue. Without the ref attribute, VueJs cannot know where to access to the tag.

NOTE: The original type of the this.$refs instance is { [key: string]: Vue | Element | Vue[] | Element[] }, while we cast it to any. This is to avoid build failure due to the linting error caused by using the original type and accessing to the resource by referencing like this.$refs.video. If we don’t want to cast it to any, we can use this.$refs["video"] instead.

We’ve so far completed the coding part. Now, let’s build this up and run a local IIS Express, and access to the web app through http://localhost:port. It works fine.

This time, instead of localhost, use the IP address. If we want to remotely access to our local dev website, this post would help.

It says we can’t use the camera because of its insecure access. In order to use the getUserMedia() API, we should use HTTPS connection to prevent private data exposure. This only happens when we’re using Google Chrome, not FireFox or Edge. So, just change the connection to HTTPS.

Now we can use IP address for camera access. Once we allow it we can immediately see our face directly on the web like below (yeah, it’s me! lol).

Let’s try this from our mobile devices. The first one is taken from Android phone, followed by the one taken from Windows Phone, then the ones from iPhone. Thanks Boris for help take those pictures!




Errr… what happened on iPhone? The camera is not accessible from both Safari for iOS and Chrome for iOS!!

This is because not all mobile web browsers support the getUserMedia() API.

getUserMedia Browser Compatibility

Here’s the data sheet from http://mobilehtml5.org/.

Unfortunately, we can’t use the getUserMedia API on iOS for now. For iOS users, we have to provide alternative methods for their user experience. There’s another API called HTML Media Capture that is supported by all mobile web browsers. It uses the traditional input type="file" tag. With this, we can access to camera on our mobile devices.

In the next post, we’re going to figure out how to provide a fallback option, if getUserMedia() API is not available.

Category:
Application Development and Integration
Tags:
, , , , ,