Developing Custom Connectors for Flow

Ahmad Najjar

0 comments

In this eBook companion to the video “Developing Custom Connectors for Microsoft Flow”, we will learn how to go beyond the tooling to solve more sophisticated scenarios and empower citizen developers. Building custom connectors to integrate directly with services, extending the server-side logic with code plugins or using webhooks.

Microsoft Flow

With Microsoft Flow, you can build modern business apps in a much reduced time since we actually empower business users, and Citizen Developers, to build their own apps rather than relying on developers as we would with more traditional development.

The whole idea of Microsoft Flow and the wider Power Platform is to empower the line of business users. These are the users who have business problems, understand their business needs, and who can pick up the development of business solutions, using Power BI, Flow and PowerApps, without having to write code, or writing a minimal amount of code.

We can also enable developers to extend. The tooling available through the Power Platform makes it possible to quickly customise Office 365, Dynamics 365 or standalone apps whilst writing minimal code. This, however, does not mean that you are limited to what you can achieve.

You can go beyond the tooling to solve more sophisticated scenarios and to empower citizen developers at every step. This could be using Custom Connectors to integrate directly with your own services, to extend server-side logic of the Common Data Service with code plugins.

This is just one of the aspects that you can extend within the Power Platform.

What is a Connector?

A connector is a wrapper around an API that allows the underlying service to talk to Microsoft Flow and PowerApps. Everything you see within Microsoft Flow is an API, a service which runs a piece of code on the server-side. So when you pass something to the service, it will execute some logic and usually return some data. The Connector makes it possible for the underlying service to talk to Microsoft Flow and PowerApps.

We have cloud and on-premises connectivity with over 260 services which we can connect to. In order to connect to on-premises data sources, we can use the Data Gateway to provide hybrid connectivity. The data gateway sits on a server on your domain, which when given a connection account will allow access to your on-premises data through the secure data gateway.

With custom connectors, you can extend the reach of your data within your organisation in order to expose it in the manner that you want.

The Architecture of a Connector

If we create a connector with Flow, there is an API Management Gateway which is provided by Microsoft. This APIM Gateway contains the definition of the connector and also a token store to manage the authentication against the API. So when we create the connection from with our Flow, we create a unique connection ID, we provide the credentials which are then passed through the gateway to the API which is hosting our service. This, in turn, will return the requested data back to our Flow or to PowerApps.

The on-premises data gateway works in exactly the same way, with the only difference being that the request passes through the gateway first of all being hitting the API endpoint which is provided by an on-premises service.

Why build a connector?

Enterprise developers may be looking for connectivity to a service which is not yet supported to Microsoft Flow, or want to connect to a custom service. It is a common scenario that you may wish to expose data from your own services whilst using logic within Microsoft Flow.

The other area is partners and ISVs to expand the number of services that you integrate with, providing the extensibility that your customers need, increasing exposure and adoption. So when you expose your data, it makes it possible for your partners to connect to your data in a secure and easy way using Microsoft Flow.

What to consider before building a Connector

Before you create your connector there are considerations which you should address before building.

Understand the API for the service which you are trying to build against to ensure that you are aware of the authentication method required as well as whether you can build triggers or if you are only able to build actions.

It is important to understand how you are going to create your connector, if you need to build it from scratch or if you have the ability to import the connector details from elsewhere. You can import OpenAPI definition files, more commonly known as Swagger files, from your local machine or from a URL. Therefore if your service provider has this file available, then you may be able to do this rather than build it yourself. Alternatively, you can build the API calls within Postman (www.getpostman.com) and export the collection into Flow.

Ensure that you set up the correct authentication method for the API as there are several available including API Keys, OAuth 2.0 (cloud only) which includes Azure AD, or Windows (on- premises only). There may be APIs which don’t require any type of authentication, therefore you can select no authentication as a security method.

Define the operations for the connector by importing sample payloads into the connection configuration screen. You will need these sample payloads as the custom connector creator will process the configuration and convert it into something which can be used by the end- users.

So with these considerations in mind, let’s create a connector.

Creating a Connector

The first thing that we are going to do is build a custom connector from scratch, and we do this through the Microsoft Flow Portal. When we want to create or edit our custom connectors, we need to expand the data menu on the left-hand menu and then select “Custom connectors”, or alternatively we can go to

Once you are in the Custom Connectors screen, you will see any custom connectors which you have created in the past. From here you have a number of options available to you, you can create a new connector, edit an existing custom connector, or download the definition file.

There are a number of options which I have available to me when I create a custom connector, including creating a blank connector or importing from external resources such as a Postman collection.

Creating a Custom Connector from Blank

Within this demo for creating a custom connector, we will be connecting to a service called Yelp. Yelp is a service which gives you information about a point of interest, for example, restaurants or shops within a given point of interest.

The first thing we need to do is give the custom connector a name. The naming convention purely comes down to your organisational standards, however, I am going to name it FVS (Flow Virtual Summit) followed by an underscore and then service name i.e. Yelp.

Once I have named the service I can click continue and start to work my way through the various screens to build the connector.

General Information

On the general information page, I can define the basic identity information for the Custom Connector. I can upload a custom image which will be displayed against the connector when my end users browse to the connector.

I can provide a background image for my icon if it has a transparent background, so I can define a HEX code colour, exactly the same as what you use in Cascading Style Sheets.

If my data source is an on-premises data source, I have the ability to check the “Connect via on- premises data gateway”.

The most important aspect of this page though is the host and the base URL. These two fields define exactly where the service is being hosted. So if I refer to the Yelp API documentation, I can find out where all of my service endpoints reside. In the case of Yelp, my endpoints are all available from https://api.yelp.com/v3.

So when I use this within my general information, I simply need to define what the host is by entering api.yelp.com. When doing this, ensure that you have selected the correct scheme, i.e. whether it is HTTPS or not, otherwise certificate errors could potentially prevent you from establishing a successful connection.

Once I have finished defining my general information, I can then click the next arrow to take me to Security.

Security

With security, I am going to define the method of authentication for communicating with my API endpoint. As previously mentioned, all common methods to authenticating are available for us to use such as API key, Oauth, Basic Authentication and even No Authentication. In the scenario where we are connecting to Yelp, the authentication method we need to use is API key as that is defined within the documentation.

When selecting API Key as our method of authentication, the form expands to prompt us for additional information such as a parameter label, parameter name and a parameter location.

First of all, it is worth setting the parameter location, as different APIs receive the data in different ways. Some can receive the key as a query string parameter, i.e. it is part of the full URL which is used to access the service. Alternatively, they can be encoded into the header of the service request. In the case of Yelp, we need to include the API key within the header of the call.

The Parameter Label is purely a label, it does not have any functional impact on the connection being established therefore you can name this something meaningful e.g. API_KEY. The more important one is the parameter name, as this is what the service endpoint will be looking for to contain the API key. In the documentation provided with Yelp, the parameter which the service is looking for is Authorization, therefore this is what I will place within the field.

Once the security has been configured, I can then move onto the next step which is to define the connection.

Definition

The definition is where we can define the triggers and actions which are associated with our custom connector. Consider any of the built-in connectors, generally they will have a trigger from which you can initiate a Flow, and then will have a number of actions where you can then interact with the service. We can define exactly the same for our custom service.

A trigger is effectively a webhook which responds to an event e.g. a file being created within SharePoint. However, what we are going to concentrate on here is the ability to create an action.

Actions and Triggers can both be created from the left hand navigated within the configuration window by clicking New action or New trigger. In this demo, we’re going to create a new action which will allow us to search for a business within the Yelp service.

The summary is what your users will see when they are going to select your action from with the Microsoft Flow action selector, therefore should be short and descriptive of the action.

More detailed information can be added to the Description which will be displayed below the action name. Finally, we have the Operation Name which should be a single word with camel case, i.e. BusinessSearch. This is what will be more widely used through PowerApps, as our connector can be used from with Microsoft Flow, Azure Logic Apps and PowerApps.

The next thing which we need to define is the request which we’re going to make against the web service. To achieve this I just need to press the Import from sample link to start building the definition.

When we press the link, a blade will appear from the right-hand side of the screen asking us for specific information about the definition, with the first being the verb. There are several verbs available to us including Get, Post, Patch etc. In the case of performing our business search, we need to do a Get against the web service to retrieve our data.

Once again I need to refer to the API documentation to understand what needs to go within these fields, and in this case, I have an endpoint specifically for being able to search. The Yelp API guide also states that I can pass additional parameters along with the request e.g. term, which is what we want to search for.

https://api.yelp.com/v3/businesses/search? term=term&location=string&latitude=1.1&longitude=1.2

Whilst it appears that I am entering a working URL here, what we paste into the URL field only needs to be a representation of the data so that the import process can understand what to ask for.

What this import process has done is to break out each aspect of our URL into its constituent parts. You can see that it displays the verb, it shows the base URL, and then it breaks down each aspect of the query string into editable objects under the Query header.

I can click the ellipsis next to the query string parameter e.g. term, and then configure how that parameter should be surfaced to the end-user. For example, I can define if the parameter is required, therefore it must have a value provided by the end-user to use this action. I can define a default value.

I can also provide a number of visibility options which define whether this is a parameter which I want my users to interact with or whether it is going to be set through the connector definition. None means that no configuration is employed, which will, therefore, default the parameter to being “important” which means it will be displayed within the main action configuration in Flow. Advanced means that it will appear on the Show advanced settings link. Internal means that your users won’t be prompted to define a value, therefore if using this option you should ensure that you have defined a default value to ensure that a value is being passed within the call.

When I return to my main definition screen, any parameters which have been identified as being required will have a red asterisk next to them.

The next thing we need to define is the response, which is the message that we’re expecting to receive back from the web service. Again, with most API documentation there is an example of what you would expect to receive from a successful call, so you can use this to paste into the Response body sample box.

When the same data has been processed, it will present the dynamic content which we expect to see with Flow.

Before we publish our connector, we should test it locally to ensure that our calls are working and that we’re retrieving the information.

Test

Before we can run a test, we need to select the link to create the connection i.e. we’re creating the link between our connector and the service endpoint.

Before we can do anything though, we need to create the connection by clicking New connection. This will begin setting up the authentication method which we defined earlier in the wizard, and so you will be prompted to insert your API key.

We then need to provide the call with all of the parameters which we defined through the query. We can then press the Test button to provide the call against the web service, and then observe what is returned.

Now that we have defined the custom connector, we need to be able to use it within our Flows.

Using the Custom Connector

To demonstrate how to use our custom connector, I am going to create a Flow which is using one of the instant triggers, specifically the “From Microsoft Flow” trigger. The first thing I am going to do is to tell the trigger that it needs to collect some information, in this example it is going to be the point of interest (POI).

I can then add another action and use the Custom Connectors filter to select only the custom connectors which have been created within my environment. Notice that our FVS_YELP connector is now displayed there. When I select that connector, you will then see the action which we have defined.

When I configure the action, you will see that it requests the parameters which I defined within the connection wizard.

We can provide it with the information we need and then test the Flow to see the JSON payload that is returned.

The great thing about creating custom connectors is that we can also consume the connector from with PowerApps to utilise the exact same functionality. We simply add our custom connector as a data source and then we can expose the actions for our app makers without doubling up effort.

Creating a Custom Connector from an Azure Service

Whilst this feature is currently in preview, we have the ability to create a connector directly from an Azure service. Just like our other connector we first of all need to give it a name, however, we then need to select the Azure subscription which contains the service.

The key thing to note is that the Azure subscriptions that you see are those that belong to the user you have used to log into Flow. Therefore, if you don’t have any subscriptions available to your user then you will not be presented with any options.

If I have successfully connected to Azure, then I will be able to choose a service. The services are broken down into two categories:

  1. Azure Functions
  2. Logic Apps

When I select the category, it will then allow me to select the relevant resource.

It is worthwhile noting that only functions which have been created with v1.0 of the Azure Function runtime are supported at this time.

Once I click continue, I am then presented with the same creation wizard as when we created from scratch, so I need to define the logo and background colour. The difference, however, is that the host, the security and the definition are already populated for us.

The key activity which you need to undertake at this point is to ensure that the actions and parameters are all named in a way which is understandable by the users.

The only part of the definition which is not set by creating a connector using this method is the request body. To rectify this you should take an example of the body which is sent to the request with your Azure Function, and then recreating the request. This will then give you the option to add the body definition.

Finishing up your Connector

Once you have got your connector up and running, there are a few things which we can do to add that final layer of polish. This will ensure that when your end users are consuming the connector and the actions, that they understand the connector is doing and what it requires from them to work.

  1. Add descriptions and friendly names – these help the users to communicate with the connector, to understand what the operation does, and to ensure that they are giving the correct information into the parameters
  2. Define more operations – expand the operations so that if you are posting data, you are then following up with a get so that you can return data back to your users
  3. Use dynamic values and dynamic schema – this will provide a better experience for your users when using the designer
  4. Add triggers – use webhooks or polling to be able to build Flows which can initiate from the service endpoint which you have connected to

Custom Connector to Connector

If you believe that your custom connector is something could add value to every else, you can share your connector with others. This goes through the normal application lifecycle, i.e. you develop it, test it, and then share it within your organisation. You then have the ability to certify your connector which makes it available to all Microsoft Flow and PowerApps users.

Summary

We have gone into a lot of depth about how custom connectors work and how we can create them within the Power Platform.

Custom Connectors allow us to bridge the gaps between services that are not already being served by Microsoft provided connectors. They allow us to connect to APIs which are both accessible by the cloud, and also to on-premises data sources by connecting through an on-premises data gateway. They provide the ability to extend the capabilities of Flow and PowerApps and help to drive adoption by simplifying the process of connecting to data sources.

Before we build a custom connector, we should understand the API to which we’re trying to connect to, specifically around the methods of authentication and the endpoints to which we’re going to connect in order to create our actions.

Once we understand the API, we can create a custom connector in a number of different ways, whether that is from scratch, an OpenAPI file, Postman or by consuming a service within Azure. If we are consuming a service such as Azure Functions, then we need to ensure that the function has been created using the V1.0 runtime.

Regardless of how we create the connector, we will always define the name of the connector and the general information such as the name, icon and background colour for the icon. We also define the host, which is the URL where the endpoints exist.

We will define the authentication method for our connector to communicate with the API, which could range from No Authentication through to OAuth 2.0 communicating with Azure AD.

We then have the ability to define triggers and actions for the API by providing samples of the API calls. These calls are processed by the connector creator and then presented as objects which we can interact with e.g. parameters allow us to define them as being required, or provide configuration about where they appear with the Flow designer.

The final step is to test the connection within the connector creator to ensure that it authenticates and returns the data which we are expecting. Once that is done, we can then test within Flow itself.

You can submit your connector to be certified by Microsoft, which means that it will then be available to users globally to interact with your services.

0000-00-00 00:00:00


Leave a Reply

Your email address will not be published. Required fields are marked

{"email":"Email address invalid","url":"Website address invalid","required":"Required field missing"}