Limit device discovery by Homes

[ArnyminerZ]

Is your feature request related to a problem? Please describe.
When multiple homes exist in the Google Home app, this integration discovers devices from all connected homes.

Describe the solution you'd like
Ability to limit google device discovery to a particular home if desired.

Additional context
Related to leikoilja/ha-google-home#363

[ArnyminerZ]

Not sure where to get the home that the device is in from, but some documentation is at Device or QueryResponse

If anyone has a Homegraph sample response it'd be easier to implement.

[ArnyminerZ]

Just for reference. By using the following script I was able to fetch the Homegraph API response for my Google Account:

from glocaltokens.client import GLocalAuthenticationTokens

client = GLocalAuthenticationTokens(
  username="...",
  password="..."
)

print("\n[*] Fetching homegraph...")
homegraph = client.get_homegraph()

An example of a Google Device is:

  devices {
    device_info {
      device_id: "00000000-0000-0000-0000-000000000000"
      agent_info {
        api_project_id: "google.com:api-project-000000000000"
        unique_id: "00000000000000000000000000000000"
      }
    }
    device_name: "Google Home"
    device_type: "action.devices.types.SPEAKER"
    traits: "action.devices.traits.Assistant"
    traits: "action.devices.traits.RemoteDucking"
    traits: "action.devices.traits.Cast"
    traits: "action.devices.traits.CommunicationCall"
    traits: "action.devices.traits.CommunicationVideoCall"
    suffix: "k"
    message12 {
      num2: 1
      num4: 1
      num5: 1
      num37: 1
      num38: 1
      num45: 1
      num46: 1
    }
    message15 {
      num5: 1541774179655
      num6: 3
      num7: 1
    }
    hardware {
      model: "Google Home"
    }
    message18 {
      device_name: "Google Home"
    }
    timestamp19: 1541774179655
    message20 {
      message1 {
        key: "communicationCallCapabilities"
        value {
          message6 {
            message1 {
              message5 {
                message1 {
                  capability: "pickup"
                  message2 {
                    bool4: true
                  }
                }
                message1 {
                  capability: "hangup"
                  message2 {
                    bool4: true
                  }
                }
                message1 {
                  capability: "reject"
                  message2 {
                    bool4: true
                  }
                }
                message1 {
                  capability: "muteMic"
                  message2 {
                    bool4: true
                  }
                }
                message1 {
                  capability: "hold"
                  message2 {
                    bool4: true
                  }
                }
              }
            }
          }
        }
      }
      message1 {
        key: "truncatedLocalNetworkId"
        value {
        }
      }
      message1 {
        key: "communicationVideoCallCapabilities"
        value {
          message6 {
            message1 {
              message5 {
                message1 {
                  capability: "pickup"
                  message2 {
                    bool4: true
                  }
                }
                message1 {
                  capability: "hangup"
                  message2 {
                    bool4: true
                  }
                }
                message1 {
                  capability: "reject"
                  message2 {
                    bool4: true
                  }
                }
                message1 {
                  capability: "muteMic"
                  message2 {
                    bool4: true
                  }
                }
                message1 {
                  capability: "turnOffCamera"
                  message2 {
                  }
                }
                message1 {
                  capability: "swapPicture"
                  message2 {
                  }
                }
              }
            }
          }
        }
      }
    }
    message25 {
      bool1: true
      bool2: true
    }
    linked_users {
      email_address: "[email protected]"
    }
    linked_users {
      email_address: "[email protected]"
    }
    local_auth_token: "..."
    states {
      name: "artist"
    }
    states {
      name: "title"
    }
    states {
      name: "subtitle"
    }
    states {
      name: "activityState"
    }
    states {
      name: "playbackState"
      value: "\020\006"
    }
    states {
      name: "mediaState"
      value: "\n\022\010\000\022\000\032\000 \000*\0002\000:\000H\006P\000"
    }
    states {
      name: "online"
      value: "\n\004true"
    }
    message30 {
      key: "mediaState"
      message2 {
        string1: "activityState"
        message2 {
          bool4: true
        }
      }
    }
    message34 {
      bool1: true
    }
    num37: 2
    string42: "\001"
  }

However, it's strange, since this device, even though it has been configured in my first home, the home name shown in the Homegraph API is the one from my second home, so something is wrong around here. Some deeper investigation needs to be done. Maybe there's an issue on how the Homegraph request is performed at client.py.

This brings to me the question if it's time to switch from the ugly and bad optimized RPC API to the new REST API which I have just noticed that exists. I believe this wasn't an option back when the package was first developed, and it's a new API created by Google, so maybe we should take a look and consider migrating, since I keep finding the RPC responses ugly, frustrating and really really strange. What do you think @leikoilja ?

[leikoilja]

Nice investigation, @ArnyminerZ 🚀
hmmm that is indeed strange that the home name shown in the homegraph api is from your second home 🤔 i wonder if we are missing setting up some kind of a scope that allows to limit the devices to a specific home, but yeh, you are right, we need to dig deeper to understand what is going on there.

Regarding the new REST API... wow, that would be so amazing to switch to that instead of the bloated RPC. However, last time i quickly skimmed through google documentation it didn't seem like things are well documented and easy to migrate, so we might stumble across a few issues while migrating. But if someone would be up for taking it for a spin, that would significantly lighten up this package :)

[ArnyminerZ]

Okey, after some investigation on the REST API I have found that Google recommends using one of their "per-language" SDKs, so in our case it'd be the Python's SDK, which can be found here. And we can use the sync method to get the same devices info as we have been getting before.

I just don't know if this is a better option than the RPC, or if it'd help with the homes organization, some testing is still needed. I'm writing a Python script to test this out, but authorisation works a little different here, so maybe the API change doesn't fit our needs, since as far as I have found for now, a Google Cloud project is needed, something that wasn't required before, and I would not like it to be required, since it'd make the integration far harder to configure.

Whatsoever, I will keep digging deeper on how auth works, and if we can simplify this process.

[ArnyminerZ]

Okey, I found a way that might be the one to go on over authorisation, however, it seems to have issues with Homegraph API authorisation. As far as I have found out, it seems like the Homegraph API is like a restricted one, so when you ask the user for permission the following error gets returned:

Error 400: invalid_scope
Some requested scopes cannot be shown: [https://www.googleapis.com/auth/homegraph]

The code that I'm using is:

import google.oauth2.credentials
import google_auth_oauthlib.flow

# client_secret.json should be a general Google Cloud secret for all the users, not every user must create its own.
flow = google_auth_oauthlib.flow.Flow.from_client_secrets_file(
   'client_secret.json',
   scopes=['https://www.googleapis.com/auth/homegraph'])

flow.redirect_uri = 'https://www.example.com/oauth2callback'

authorization_url, state = flow.authorization_url(
    # Enable offline access so that you can refresh an access token without
    # re-prompting the user for permission. Recommended for web server apps.
    access_type='offline',
    # Enable incremental authorization. Recommended as a best practice.
    include_granted_scopes='true'
    )

print(authorization_url)

Will keep researching on how to fix this, or ask the user for auth in another way.

[ArnyminerZ]

Okey, got some big news on the authorisation side. I've managed to request permission to the user successfully for accessing the resources of their Google Cloud Account, which should help on automating the requests required to set up the user's account to use the Homegraph API, since as far as I have found a GCloud project is needed, even though I'm not sure. Whatsoever, I've managed to get an access token for the user's account successfully with a right now quite dirty code.

First I get an authorisation URL from the following code:

import google.oauth2.credentials
import google_auth_oauthlib.flow

# client_secret.json should be a general Google Cloud secret for all the users, not every user must create its own.
flow = google_auth_oauthlib.flow.Flow.from_client_secrets_file(
   'client_secret.json',
   scopes=['https://www.googleapis.com/auth/cloud-platform'])

# Returns as get parameters:
flow.redirect_uri = 'https://www.example.com/oauth2callback'

authorization_url, state = flow.authorization_url(
    # Enable offline access so that you can refresh an access token without
    # re-prompting the user for permission. Recommended for web server apps.
    access_type='offline',
    # Enable incremental authorization. Recommended as a best practice.
    include_granted_scopes='true'
    )

print(authorization_url)

Please note that I've set up my own personal GCloud project for doing all of this, but in this way there will be only one Google Cloud project for all the user's codebase, so for the final user this doesn't matter a lot. And I've put the authorisation keys for the project at client_secret.json.

And then, with the following code I extract the authorisation credentials that should work for all of our requests to the Python's Homegraph API, whose example I will post ASAP. Take a look at this:

from googleapiclient.discovery import build
import google.oauth2.credentials
import google_auth_oauthlib.flow

response_url="https://www.example.com/oauth2callback?state=...&code=...&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcloud-platform"

def credentials_to_dict(credentials):
  return {'token': credentials.token,
          'refresh_token': credentials.refresh_token,
          'token_uri': credentials.token_uri,
          'client_id': credentials.client_id,
          'client_secret': credentials.client_secret,
          'scopes': credentials.scopes}

flow = google_auth_oauthlib.flow.Flow.from_client_secrets_file(
   'client_secret.json',
   scopes=['https://www.googleapis.com/auth/cloud-platform']
)
flow.redirect_uri = 'https://www.example.com/oauth2callback'
flow.fetch_token(authorization_response=response_url)

credentials = credentials_to_dict(flow.credentials)

print(credentials)

The response_url is the URL where you are redirected after getting authorised at the Google's page, so you just have to copy and paste this url into the variable, and everything should work.

At the end you get a dict with the following format:

{
  "token":"",
  "refresh_token":"",
  "token_uri":"",
  "client_id":"",
  "client_secret":"",
  "scopes":[]
}

---

Reference (mostly):

• https://cloud.google.com/apis/docs/client-libraries-explained
• https://github.com/googleapis/google-api-python-client/blob/main/docs/start.md
• https://developers.google.com/identity/protocols/oauth2/web-server#example
• https://googleapis.github.io/google-api-python-client/docs/dyn/cloudresourcemanager_v3.projects.html#create
• https://requests-oauthlib.readthedocs.io/en/stable/api.html#requests_oauthlib.OAuth2Session.fetch_token
• https://google-auth-oauthlib.readthedocs.io/en/latest/reference/google_auth_oauthlib.flow.html#google_auth_oauthlib.flow.Flow

[leikoilja]

Those are some amazing findings and progress, @ArnyminerZ 🚀
I m just wondering if the tokens are the same between getting them this way through Google Cloud Platform and the legacy way as we currently do it? Could it be possible to get the tokens as we do now (without the need to create a new project at google cloud) and use those tokens to call REST API? 🤔

[ArnyminerZ]

I don't know, can be tested. I'd wait to implement new login methods in the future if we find it necessary, but as you've stated, maybe requesting through the REST API can be done with the credentials we already got. Will check asap. 😉

[rithvikvibhu]

Haven't tried it yet, but it looks like the official homegraph API (REST or RPC) is different from the one used by the app.

@ArnyminerZ's links:
REST: https://homegraph.googleapis.com/v1/
RPC: google.home.graph.v1.HomeGraphApiService

Unofficial one used by GH app:
RPC: google.internal.home.foyer.v1.StructuresService

The official one gives limited info and only fields exposed by the device manufacturer (color, temperature, power, etc.) and not meta details about the device itself (which is what's needed to get the local auth token).

[ArnyminerZ]

Yeah, I understand, so we should keep using the same API 😄