Google Photos: add support for supplemental-metadata.json files

[simulot]

Discussed in #650

^{Originally posted by phands January 24, 2025}
Hi all,

Sorry if this is a daft question.
I have a large-ish (just over 202Gigs) Takeout directory (extracted from the compressed archives to a directory structure), and I'm trying to use immich-go to upload it all.

It seems to work partially, but I only see a tiny fraction of the files in the immich server. There are tens of thousand of photos and videos. but less than 25 in Immich.

I know that the uploader is changing rapidly, and I'd love to help with testing if I can.
My Takeout directory is structured as shown in the screenshot...

Immich-go says it's seeing 31553 images...see this screenshot...

my iimmich-go command is ....
/immich-go upload from-google-photos /local-12T/Google\ Photos/Takeout --api-key 853oJrfwOpiGB8XhASoAraVsQ8USDS156Eu1nLaNWZY --server http://10.0.0.37:2283

The log file seems OK, but shows that while it scanned 31553 images, it only uploaded 35 the first time, and now they are duplicates. Here are the last few lines of 102900...

2025-01-24 12:31:34 INF Tagged file=Takeout:Photos from 2024/Simon Durrant and Anna and Alan Kidd and wife.jpg tags=[Takeout]
2025-01-24 12:31:34 INF added to an album file=Takeout:Photos from 2024/Simon Durrant and Anna and Alan Kidd and wife.jpg Album=TATC 11th Entry
2025-01-24 12:33:39 INF
2025-01-24 12:33:39 INF Input analysis:
2025-01-24 12:33:39 INF ---------------
2025-01-24 12:33:39 INF scanned image file : 31553
2025-01-24 12:33:39 INF scanned video file : 2926
2025-01-24 12:33:39 INF scanned sidecar file : 33311
2025-01-24 12:33:39 INF discarded file : 0
2025-01-24 12:33:39 INF unsupported file : 468
2025-01-24 12:33:39 INF file duplicated in the input : 7
2025-01-24 12:33:39 INF associated metadata file : 42
2025-01-24 12:33:39 INF missing associated metadata file : 34437
2025-01-24 12:33:39 INF
2025-01-24 12:33:39 INF Uploading:
2025-01-24 12:33:39 INF ----------
2025-01-24 12:33:39 INF uploaded : 0
2025-01-24 12:33:39 INF upload error : 0
2025-01-24 12:33:39 INF file not selected : 0
2025-01-24 12:33:39 INF server's asset upgraded with the input : 0
2025-01-24 12:33:39 INF server has same asset : 35
2025-01-24 12:33:39 INF server has a better asset : 0
2025-01-24 12:33:39 INF
2025-01-24 12:33:39 INF Check the log file: /home/phands/.cache/immich-go/immich-go_2025-01-24_12-31-01.log

Can anyone point me at where to start debugging or what is wrong with my immich-go command? I'm also available for live debugging (USA Eastern time).

Thanks,

Paul

[simulot]

Normal cases:

Takeout/Google Photos/Photos from 2016/20160123_181953.jpg.supplemental-metadata.json
Takeout/Google Photos/Photos from 2016/20160123_181953.jpg

Some special cases:

Screenshot_20231027_123303_Facebook(1).jpg
Screenshot_20231027_123303_Facebook.jpg.supple.json
Screenshot_20231027_123303_Facebook.jpg.supple(1).json
Screenshot_20231027_123303_Facebook.jpg
Screenshot_20231027_123303_Facebook-edited.jpg
original_c576ba15-9661-4bd4-b0bf-a7bf2ce3d354_.json
original_c576ba15-9661-4bd4-b0bf-a7bf2ce3d354_2.jpg
original_c576ba15-9661-4bd4-b0bf-a7bf2ce3d354_2(1).jpg

[phands]

OK - for Screenshot_20231027_123303_Facebook, the supplemental JSON is...

{
  "title": "Screenshot_20231027_123303_Facebook.jpg",
  "description": "",
  "imageViews": "34",
  "creationTime": {
    "timestamp": "1698424810",
    "formatted": "Oct 27, 2023, 4:40:10 PM UTC"
  },
  "photoTakenTime": {
    "timestamp": "1698424383",
    "formatted": "Oct 27, 2023, 4:33:03 PM UTC"
  },
  "geoData": {
    "latitude": 0.0,
    "longitude": 0.0,
    "altitude": 0.0,
    "latitudeSpan": 0.0,
    "longitudeSpan": 0.0
  },
  "geoDataExif": {
    "latitude": 0.0,
    "longitude": 0.0,
    "altitude": 0.0,
    "latitudeSpan": 0.0,
    "longitudeSpan": 0.0
  },
  "url": "https://photos.google.com/photo/AF1QipMd5ebd_JZYwzCVR8kmmjl8Fd8nEPYH2_GozFyQ",
  "googlePhotosOrigin": {
    "mobileUpload": {
      "deviceFolder": {
        "localFolderName": ""
      },
      "deviceType": "ANDROID_TABLET"
    }
  }
}

For a normal case, here's

Takeout/Google Photos/Photos from 2016/20160123_181953.jpg.supplemental-metadata.json

{
  "title": "20160123_181953.jpg",
  "description": "",
  "imageViews": "59",
  "creationTime": {
    "timestamp": "1463271029",
    "formatted": "May 15, 2016, 12:10:29 AM UTC"
  },
  "photoTakenTime": {
    "timestamp": "1453591193",
    "formatted": "Jan 23, 2016, 11:19:53 PM UTC"
  },
  "geoData": {
    "latitude": 40.785555599999995,
    "longitude": -73.9766667,
    "altitude": 44.0,
    "latitudeSpan": 0.0,
    "longitudeSpan": 0.0
  },
  "geoDataExif": {
    "latitude": 40.785555599999995,
    "longitude": -73.9766667,
    "altitude": 44.0,
    "latitudeSpan": 0.0,
    "longitudeSpan": 0.0
  },
  "url": "https://photos.google.com/photo/AF1QipMQNle_SmSJPywQ9UfT9oz5KrX1xU9a9xEaO3be",
  "googlePhotosOrigin": {
    "mobileUpload": {
      "deviceFolder": {
        "localFolderName": ""
      },
      "deviceType": "ANDROID_PHONE"
    }
  }
}

Would you like any others?

[simulot]

Thank you.
the file layout is unchanged.