Mixed Reality in Power Apps

Daniel Christian

0 comments

This book gives an overview of the new mixed reality feature in Power Apps. We’ll look at how we can build apps that allow users to work with 3D models, view them in mixed reality and also take 3D measurements in real time.

Introducing Mixed Reality in Power Apps

Mixed reality was announced on April 1st, 2020. As well as providing 3D mixed reality, it also provides some other uses such as measuring spaces in real-time, for example measuring up a room for new furniture. It can also be used to take linear or volumetric measurements.

3D models can also be used to better communicate changes to a workspace or show where new equipment will fit in, allowing better communication.

When it was first released in Power Apps, the mixed reality feature was experimental, but it has now moved on to become a preview feature. In order to use it in your apps, you need to make sure that the option is turned on in the Power Apps settings:

3-Dimensional Views

When starting to work with mixed reality and 3D views in Power Apps, you may be wondering where you can get some sample models. Luckily enough, if you have Windows 10, then you already have some available. Search for the Microsoft 3D Viewer app on your device, and you will be able to access a number of example models:

The file format for 3D models used in Power Apps is .glb, and these files can be stored in SharePoint, OneDrive or GitHub for use in your app. In the future, you’ll also be able to make API calls to existing 3D file locations.

Measurement

The term segment, when used in Power Apps, is used to describe a side of a shape, so for example, the image below has four segments:

Measurement properties are broken up into two sections in Power Apps:

Measurements

  • Length – Sum of all the segments in a shape
  • Unit – The unit of measurement (centimetres, metres etc.)

Measurement Details

  • Distance – Total distance of each segment
  • Direction – Vector direction of each segment
  • X, Y & Z – Direction of each segment

Creating Mixed Reality Apps

3D Model App

As well as the 3D viewer app mentioned earlier, there is also another app on Windows 10 called 3D Builder. Using this app, you can select from a large number of 3D models in various categories:

These models can then be saved in the .glb format for use in your Power App:

In this example, the 3D models being used are saved in a OneDrive folder, which is shown below:

You’ll notice that there is also a spreadsheet file saved in the same folder, which is what Power Apps uses to pull the models into the app. The structure of the spreadsheet is shown below:

The full path for the files isn’t used, and the .\ notation is used instead as the spreadsheet is in the same location as the 3D models. Also, note that the [Image] text is required in the path column header as otherwise, Power Apps will only reference this as a link rather than the image itself.

To use these images in our app, we need to add OneDrive as a data source, then browse to the folder and select the spreadsheet:

The next steps we’re going to do is add a horizontal gallery and reference our 3D Models data source:

In order to easily see which item is selected in the gallery, we can add the following to the following to the gallery TemplateFill attribute, which will change the colour of the currently selected item:

If(ThisItem.IsSelected,GreenYellow,RGBA(0, 0, 0, 0))

The next thing to add to the app is View in 3D, which is available under the Media dropdown in the Insert menu:

In order to show the 3D models in this control, we’ll change the Source attribute to the following:

Gallery1.Selected.’3DModel’

When you do this, you may get the following error in the control:

However, if you go into preview mode and select a 3D model in the gallery, this error should disappear. Please note, the first time you do this, the control can take a little while to display the model:

Eventually, the model will load and then any subsequent models that are selected will display straight away. It’s just the first time that it can take some time.

View in Mixed Reality App

The look and feel of our example mixed reality app are the same as the previous app and consists of a horizontal gallery containing some 3D models along with a View in MR button:

However, if you go into preview mode on a computer and click the View in MR button, you’ll get the following error:

This makes sense as the only way to test and use the app is to do so on a device that is capable of showing mixed reality. To show this, we’ll open the Power Apps mobile app on a phone, select the gazebo 3D model and then press the button.

The first thing we’ll need to do then is to point our mobile device at a surface and select somewhere to place the object:

This will then display the 3D model in your location, allowing you to move around it and view it from multiple angles, as if it were in the room with you:

It is also possible to take photos with the 3D model for you to view later on.

In our app design view the way we added the View in MR button was to go to the Mixed Reality dropdown in the Insert menu and then select View in MR (preview):

We then just add the Source attribute to be the selected item in the gallery.

Measure in Mixed Reality App

The final example app shows how to use mixed reality to measure shapes in real-time:

Pressing the Length button gives us another button:

This opens the mixed reality screen where we can press once to choose the first point of measurement and then tap again on the second point to show the distance between the two:

Once we’re done, we press Submit, and then a summary of the measurement is displayed:

The area of a shape can also be measured. This time though, we are prompted to select all of the sides of the shape we want to measure:

The next measurement we can take is the volume of a shape. As volume requires three dimensions of measurement though we measure the sides of the shape first, as we did with the area, but then it asks for the height of the shape:

Also note in this example, we’re using inches as the unit of measurement. As with the other measurements, we then get a summary of the measurements taken:

Now we’ll look at how we build the above functions in Power Apps. In the length example, it is made up of a button, galleries to display the measurement results, and an image display to show the picture of the measured shape:

The button is added from the Mixed Reality dropdown in the Insert Menu. Pressing the Measure Distance in MR allows the user to add the data by measuring in mixed reality and then this data is stored in the galleries.

Real-Life Use Case for Mixed Reality

The examples shown so far are fun and show how mixed reality can work in Power Apps, but what about a real-world scenario? The example below shows an inventory app for an air conditioning company:

Most of this app is fairly straightforward, showing client locations and product information.

One screen shows a gallery of all available products, which again is fairly standard. However, when a user selects a product, there is a toggle to switch between a 2D picture of the product and a 3D model:

Additionally, the 3D model of the unit can then be displayed in mixed reality so the salesperson and client can see exactly how it will look when installed:

0000-00-00 00:00:00


Leave a Reply

Your email address will not be published. Required fields are marked

{"email":"Email address invalid","url":"Website address invalid","required":"Required field missing"}