Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Custom metrics #8

Merged
merged 5 commits into from
Feb 29, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions Readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ A lot of these samples require an [Azure subscription, click here to get started
| [Linux metrics with Telegraf and Azure Monitor](linux_metrics_telegraf_azuremonitor) | This script installs and configures [Telegraf](https://github.com/influxdata/telegraf) to send VM performance metrics (CPU/memory/disk/network) to [Azure Monitor](https://learn.microsoft.com/en-us/azure/azure-monitor/overview). |
| [Linux logs with Fluent Bit and Azure Data Explorer (Kusto)](linux_logs_fluentbit_kusto) | This script installs and configures [Fluent Bit](https://github.com/fluent/fluent-bit) to capture real-time logs from your game servers and send them to [Azure Data Explorer (Kusto)](https://learn.microsoft.com/en-us/azure/data-explorer/data-explorer-overview). |
| [Linux logs with Fluent Bit and Azure Blob Storage](linux_logs_fluentbit_azurestorage) | This script installs and configures [Fluent Bit](https://github.com/fluent/fluent-bit) to capture real-time logs from your game servers and send them to [Azure Blob Storage](https://learn.microsoft.com/en-us/azure/storage/blobs/storage-blobs-introduction). |
| [Linux custom game server metrics](windows_linux_custom_gameserver_metrics) | This script shows you how to send metrics from your game server to [Telegraf](https://github.com/influxdata/telegraf) and then to the backend of your choice |

### Windows

Expand All @@ -30,6 +31,7 @@ A lot of these samples require an [Azure subscription, click here to get started
| [Windows metrics with Telegraf and Application Insights](windows_metrics_telegraf_applicationinsights) | This script installs and configures [Telegraf](https://github.com/influxdata/telegraf) to send VM performance metrics (CPU/memory/disk/network) to [Application Insights](https://learn.microsoft.com/en-us/azure/azure-monitor/app/app-insights-overview). |
| [Windows logs with Telegraf and PlayFab](windows_logs_telegraf_playfab) | This script installs and configures [Telegraf](https://github.com/influxdata/telegraf) to grab game server logs and send them to [PlayFab](https://learn.microsoft.com/en-us/gaming/playfab/). |
| [Windows enable optional diagnostics](windows_enable_optional_diagnostics) | This script enables sending diagnostic data notify Microsoft of application faults, kernel faults, unresponsive applications, and other application specific problems. |
| [Windows custom game server metrics](windows_linux_custom_gameserver_metrics) | This script shows you how to send metrics from your game server to [Telegraf](https://github.com/influxdata/telegraf) and then to the backend of your choice |

## Debugging

Expand Down
41 changes: 41 additions & 0 deletions windows_linux_custom_gameserver_metrics/PF_StartupScript.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
<#
.SYNOPSIS
<Overview of script>
.DESCRIPTION
<Brief description of script>
.NOTES
Version: 1.0
Author: <Name>
Creation Date: <Date>
Purpose/Change: Initial script development
#>

# Set Error Action to Stop
$ErrorActionPreference = "Stop"

# Grab the script path
$scriptPath = split-path -parent $MyInvocation.MyCommand.Definition
echo "the current script path is $scriptPath"

# these are some of the env variables that are available to you
echo "PlayFab Title ID is $env:PF_TITLE_ID" # e.g. 59F84
echo "PlayFab Build ID is $env:PF_BUILD_ID" # Guid, e.g. 09d91059-22d3-4d0a-8c99-f34f80525aae
echo "PlayFab Virtual Machine ID is $env:PF_VM_ID" # e.g. vmss:SouthCentralUs:2458795A9259968E_12fe54be-fae1-41aa-83d9-09b809d5ef01:09d91059-22d3-4d0a-8c99-f34f80525aae
echo "Region where the VM is deployed is $env:PF_REGION" # e.g. SouthCentralUs
echo "Shared content folder is $env:PF_SHARED_CONTENT_FOLDER_VM" # e.g. D:\sharedcontentfolder (All servers running on this VM have access to this folder through the PF_SHARED_CONTENT_FOLDER env variable.)

# configure the telegraf configuration file with proper dimensions (title ID/build ID/vm ID)
$telegrafConfPath = "$scriptPath\telegraf.conf"
((Get-Content -path $telegrafConfPath -Raw) -replace '_%PF_TITLE_ID%_', "$env:PF_TITLE_ID") | Set-Content -Path $telegrafConfPath
((Get-Content -path $telegrafConfPath -Raw) -replace '_%PF_BUILD_ID%_', "$env:PF_BUILD_ID") | Set-Content -Path $telegrafConfPath
((Get-Content -path $telegrafConfPath -Raw) -replace '_%PF_VM_ID%_', "$env:PF_VM_ID") | Set-Content -Path $telegrafConfPath

# install telegraf as a Windows service
$telegrafPath = 'C:\Program Files\telegraf'
New-Item -ItemType Directory -Force -Path "$telegrafPath"
cp "$scriptPath\telegraf.*" "$telegrafPath"
cd "$telegrafPath"
.\telegraf.exe --service install --config "$telegrafPath\telegraf.conf"

# start the telegraf service
.\telegraf.exe --service start
26 changes: 26 additions & 0 deletions windows_linux_custom_gameserver_metrics/PF_StartupScript.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
#!/bin/bash
set -o errexit # script exits when a command fails == set -e
set -o nounset # script exits when tries to use undeclared variables == set -u
set -o xtrace # trace what's executed == set -x (useful for debugging)
set -o pipefail # causes pipelines to retain / set the last non-zero status

# get script's path
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"

# install telegraf
sudo apt-get -yq --no-install-recommends install "${DIR}"/telegraf.deb

systemctl stop telegraf

# remove \r\n in-place
sed -i -e 's/\r$//' ${DIR}/telegraf.conf

mkdir -p /etc/telegraf || echo 'Warning: /etc/telegraf already exists'

# add PF_TITLE_ID, PF_BUILD_ID, PF_VM_ID to telegraf.conf as dimensions
sed -e "s/_%PF_TITLE_ID%_/${PF_TITLE_ID}/g" -e "s/_%PF_BUILD_ID%_/${PF_BUILD_ID}/g" -e "s/_%PF_VM_ID%_/${PF_VM_ID}/g" -e "s/_%PF_REGION%_/${PF_REGION}/g" ${DIR}/telegraf.conf > /etc/telegraf/telegraf.conf

# add user telegraf to group docker so telegraf daemon can read docker logs
sudo usermod -a -G docker telegraf

systemctl restart telegraf
62 changes: 62 additions & 0 deletions windows_linux_custom_gameserver_metrics/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
# Windows/Linux custom game server metrics

## Introduction

[Telegraf](https://github.com/influxdata/telegraf) is a plugin-driven server agent for collecting and reporting metrics. It supports a wide variety of inputs, processors, aggregators, and outputs.

## What it does

This script uses [telegraf](https://www.influxdata.com/time-series-platform/telegraf/) agent that sets up an HTTP server to listen for external metrics. Using this solution, you can send any kind of metrics from your game server to telegraf and you can use any of the [telegraf output plugins](https://github.com/influxdata/telegraf/blob/master/docs/OUTPUTS.md) to store them, in the backend of your choice.

The documentation for the [http listener telegraf input plugin is here](https://github.com/influxdata/telegraf/blob/master/plugins/inputs/http_listener_v2/README.md), check it out for instructions on how to properly configure it. You can also check this [telegraf.conf sample config file](./telegraf.conf) file for an example of how to configure telegraf to listen for metrics on port 8080.

To send custom metrics, you need send a POST request to the telegraf server from your game server. To do that, you need to find the host VM IP address. This is different depending on the type of Build you are using.

### Host IP using Windows or Linux Processes

If you are using [process mode](https://learn.microsoft.com/en-us/gaming/playfab/features/multiplayer/servers/deploy-using-game-manager#server-details-for-process-mode), you can talk to telegraf using `localhost` or `127.0.0.1`.

### Host IP using Windows Containers

PlayFab configures a bridge network between containers and the host VM. To get the IP of the host when using Windows Containers, you can use the following Powershell script from inside the container, before you start your game server:

```powershell
$gateway = (Get-WmiObject -Class Win32_IP4RouteTable | where { $_.destination -eq '0.0.0.0' -and $_.mask -eq '0.0.0.0' } | sort metric1 | select nexthop).nexthop
Write-Output $gateway
```

### Host IP using Linux Containers

PlayFab configures a bridge network between containers and the host VM. To get the IP of the host when using Linux Containers, you can run a script from inside the container, before you start your game server. The script you can run depends on the OS that you configure your container image with. For example, if you are using Ubuntu, you can do `sudo apt update && sudo apt install net-tools` to install `netstat` and then grab the host VM (which acts as a gateway) IP using:

```bash
netstat -rn | grep '^0.0.0.0' | awk '{print $2}'
```

## Sending metrics to telegraf

To send metrics to telegraf, you'd need to send the data in a specific format. You can see the documentation [here](https://github.com/influxdata/telegraf/blob/master/plugins/inputs/http_listener_v2/README.md) and a couple of samples below:

```bash
curl -i -XPOST 'http://localhost:8080/telegraf' --data-binary 'cpu_load_short,host=server01,region=us-west value=0.64 1434055562000000000'
```

```powershell
$headers = @{
"Content-Type" = "application/octet-stream"
}
$body = "cpu_load_short,host=server01,region=us-west value=0.64 1434055562000000000"
$response = Invoke-WebRequest -Uri "http://localhost:8080/telegraf" -Method POST -Headers $headers -Body $body
```

## Usage

You should download telegraf from the [GitHub releases](https://github.com/influxdata/telegraf/releases) section. The [Windows amd64 package](https://dl.influxdata.com/telegraf/releases/telegraf-1.24.4_windows_amd64.zip) works with this sample.

Then, create a zip file with the following contents:

- telegraf.conf
- telegraf package file (e.g. telegraf.exe)
- PF_StartupScript.ps1 or PF_StartupScript.sh file (depending on the OS you are using)

You can now create a new MPS Build with your startup script using the [instructions here](https://learn.microsoft.com/en-us/gaming/playfab/features/multiplayer/servers/vmstartupscript).
28 changes: 28 additions & 0 deletions windows_linux_custom_gameserver_metrics/telegraf.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
[agent]
interval = "10s"
round_interval = true
metric_batch_size = 1000
metric_buffer_limit = 10000
collection_jitter = "0s"
flush_interval = "60s"
flush_jitter = "0s"
precision = ""
debug = true
omit_hostname = true

[global_tags]
titleID = "_%PF_TITLE_ID%_"
buildID = "_%PF_BUILD_ID%_"
vmID = "_%PF_VM_ID%_"

[[outputs.file]]
## Files to write to, "stdout" is a specially handled file.
# this is just for testing, you should add the outputs of your choice
files = ["stdout"]

# Generic HTTP write listener
[[inputs.http_listener_v2]]
## Address and port to host HTTP listener on
service_address = ":8080"
data_format = "influx"

Loading