Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create Routing_Instructions.md and routing references #42

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

george-guirguis
Copy link
Collaborator

Purpose

  • ...

Checklist

  • I have read the contribution guidelines.
  • I submitted this PR against the correct branch:
    • This pull-request is submitted against the main branch.
    • I have merged the latest main branch prior to submission and re-merged as needed after I took any feedback.
    • I will squashed my changes into one with a clear description of the change when I complete the PR.

Does this introduce a breaking change?

[ ] Yes
[ ] No

Pull Request Type

What kind of change does this Pull Request introduce?

[ ] Bugfix
[ ] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Documentation content changes
[ ] Other... Please describe:

How to Test

  • ...

What to Check

Verify that the following are valid

  • ...

Other Information

@george-guirguis
Copy link
Collaborator Author

@rido-min @veyaddan please review and merge.

@rido-min
Copy link
Contributor

I'd like to test the routing scenario first. @vaavva can you validate before merging?

Copy link
Member

@vaavva vaavva left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A lot of these commands didn't run as-is, and I couldn't complete the whole flow. It would also be great if the commands didn't require as much manual substitution and used environment variables like the rest of the repo.

# Routing Telemetry
Event Grid allows you to route your MQTT messages to Azure services or webhooks for further processing. Accordingly, you can build end-to-end solutions by leveraging your IoT data for data analysis, storage, and visualizations, among other use cases.

The routing configuration enables you to send all your messages from your clients to an [Event Grid custom topic](https://learn.microsoft.com/en-us/azure/event-grid/custom-topics), and configuring [Event Grid event subscriptions](https://learn.microsoft.com/en-us/azure/event-grid/subscribe-through-portal) to route the messages from that custom topic to the [supported event handlers](https://review.learn.microsoft.com/en-us/azure/event-grid/event-handlers).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The routing configuration enables you to send all your messages from your clients to an [Event Grid custom topic](https://learn.microsoft.com/en-us/azure/event-grid/custom-topics), and configuring [Event Grid event subscriptions](https://learn.microsoft.com/en-us/azure/event-grid/subscribe-through-portal) to route the messages from that custom topic to the [supported event handlers](https://review.learn.microsoft.com/en-us/azure/event-grid/event-handlers).
The routing configuration enables you to send all your messages from your clients to an [Event Grid custom topic](https://learn.microsoft.com/azure/event-grid/custom-topics), and configuring [Event Grid event subscriptions](https://learn.microsoft.com/azure/event-grid/subscribe-through-portal) to route the messages from that custom topic to the [supported event handlers](https://review.learn.microsoft.com/azure/event-grid/event-handlers).


The routing configuration enables you to send all your messages from your clients to an [Event Grid custom topic](https://learn.microsoft.com/en-us/azure/event-grid/custom-topics), and configuring [Event Grid event subscriptions](https://learn.microsoft.com/en-us/azure/event-grid/subscribe-through-portal) to route the messages from that custom topic to the [supported event handlers](https://review.learn.microsoft.com/en-us/azure/event-grid/event-handlers).

These instructions guide you to route your filtered MQTT messages from your Event Grid namespace to an [Azure Event Hubs](https://learn.microsoft.com/en-us/azure/event-hubs/event-hubs-about) for further processing. Consider a use case where you want to route the telemetry from only vehicle1.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
These instructions guide you to route your filtered MQTT messages from your Event Grid namespace to an [Azure Event Hubs](https://learn.microsoft.com/en-us/azure/event-hubs/event-hubs-about) for further processing. Consider a use case where you want to route the telemetry from only vehicle1.
These instructions guide you to route your filtered MQTT messages from your Event Grid namespace to an [Azure Event Hubs](https://learn.microsoft.com/azure/event-hubs/event-hubs-about) for further processing. Consider a use case where you want to route the telemetry from only vehicle1.

Configure these resources after the Event Grid namespace configuration to enable the routing flow.

### Create an Event Hubs instance
Use [these instructions](https://learn.microsoft.com/en-us/azure/event-hubs/event-hubs-quickstart-cli) to create an Event Hubs instance.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Use [these instructions](https://learn.microsoft.com/en-us/azure/event-hubs/event-hubs-quickstart-cli) to create an Event Hubs instance.
Use [these instructions](https://learn.microsoft.com/azure/event-hubs/event-hubs-quickstart-cli) to create an Event Hubs instance.

### View the routed MQTT messages in Azure Event Hubs
After you run the samples to send the MQTT messages, follow these steps to view the routed MQTT messages in Azure Event Hubs using Azure Stream Analytics query
- Navigate to the Event Hubs instance on the Azure portal.
- Go to [**Process data**](https://learn.microsoft.com/en-us/azure/event-hubs/process-data-azure-stream-analytics) using Azure Stream Analytics.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Go to [**Process data**](https://learn.microsoft.com/en-us/azure/event-hubs/process-data-azure-stream-analytics) using Azure Stream Analytics.
- Go to [**Process data**](https://learn.microsoft.com/azure/event-hubs/process-data-azure-stream-analytics) using Azure Stream Analytics.

- Configure your Event Grid Topic where your messages will be routed. The region of the topic needs to match the region of your namespace.
```bash
az eventgrid topic create
--name {EG custom topic name} \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some of these are defined in the create an event hub instance instructions - we should either use the same vars or if these are from az.env, include source az.env


```bash
az eventgrid event-subscription create --name contosoEventSubscription \
--source-resource-id "/subscriptions/$sub_id/resourceGroups/$rg/providers/Microsoft.EventGrid/topics/{Your Event Grid Topic Name}" \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we save the event grid topic name so the customers don't have to keep copying and pasting these commands to put their topic name in?

az eventgrid event-subscription create --name contosoEventSubscription \
--source-resource-id "/subscriptions/$sub_id/resourceGroups/$rg/providers/Microsoft.EventGrid/topics/{Your Event Grid Topic Name}" \
--endpoint-type eventhub \
--endpoint /subscriptions/$sub_id/resourceGroups/$rg/providers/Microsoft.EventHub/namespaces/{Event Hub Namespace Name}/eventhubs/{Event Hub Name}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
--endpoint /subscriptions/$sub_id/resourceGroups/$rg/providers/Microsoft.EventHub/namespaces/{Event Hub Namespace Name}/eventhubs/{Event Hub Name}
--endpoint /subscriptions/$sub_id/resourceGroups/$rg/providers/Microsoft.EventHub/namespaces/{Event Hub Namespace Name}/eventhubs/{Event Hub Name} \

},
"topicSpacesConfiguration": {
"state": "Enabled"
"routeTopicResourceId": "/subscriptions/{Subscription ID}/resourceGroups/{Resource Group ID}/providers/Microsoft.EventGrid/topics/{EG Custom Topic Name}"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"routeTopicResourceId": "/subscriptions/{Subscription ID}/resourceGroups/{Resource Group ID}/providers/Microsoft.EventGrid/topics/{EG Custom Topic Name}"
"routeTopicResourceId": "/subscriptions/$sub_id/resourceGroups/$rg/providers/Microsoft.EventGrid/topics/{EG Custom Topic Name}"

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nevermind, I see this has to be typed out :(

Set the routing configuration on the Event Grid referring to the Event Grid topic.

```bash
az resource update --id $res_id --is-full-object --properties '{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

which resource id should this be? I was assuming the one in my az.env, but I get this error:

unrecognized arguments: --is-full-object --properties {
  "properties": {
    "isZoneRedundant": true,
    "topicsConfiguration": {
      "inputSchema": "CloudEventSchemaV1_0"
    },
    "topicSpacesConfiguration": {
      "state": "Enabled"
      "routeTopicResourceId": "/subscriptions/$sub_id/resourceGroups/$rg/providers/Microsoft.EventGrid/topics/val-eg-custom-topic"
    }
  },
  "location": "eastus2euap"
}

Examples from AI knowledge base:
az resource update --ids /subscriptions/{SubID}/resourceGroups/{ResourceGroup}/providers/Microsoft.Web/sites/{WebApp} --set tags.key=value
Update a webapp. (autogenerated)

az resource update --ids /subscriptions/{SubID}/resourceGroups/{ResourceGroup}/providers/Microsoft.Web/sites/{WebApp} --set tags.key=value --latest-include-preview
Update a webapp by using the latest api-version whether this version is a preview version.

https://docs.microsoft.com/en-US/cli/azure/resource#az_resource_update
Read more about the command in reference docs

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this should be

Suggested change
az resource update --id $res_id --is-full-object --properties '{
az resource patch --ids $res_id --is-full-object --properties '{

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After changing to the patch command, I'm still getting an error
(InvalidRequest) Route topic val-eg-custom-topic not found for namespace mqtt-samples-eg Code: InvalidRequest Message: Route topic val-eg-custom-topic not found for namespace mqtt-samples-eg

Not sure if this is because I had the wrong resource id or should have somehow linked the event hub steps to my event grid namespace

"inputSchema": "CloudEventSchemaV1_0"
},
"topicSpacesConfiguration": {
"state": "Enabled"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"state": "Enabled"
"state": "Enabled",

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants