Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Attempted to touch a stale object: SubjectSet. #290

Open
fulsdavid opened this issue Nov 5, 2022 · 1 comment
Open

Attempted to touch a stale object: SubjectSet. #290

fulsdavid opened this issue Nov 5, 2022 · 1 comment

Comments

@fulsdavid
Copy link

Having an intermittent issue when uploading subjects via the python client. No indication that I am losing network connectivity. I spawn up to 20 processes working on preparing subjects from individual files, then all uploading to the same subject set. I basically create the subject, add the png file, get the subject id, add that to the already prepared dictionary with the manifest, then add the manifest to the subject. Once all subjects are complete, I try to add them to the proper subject set where I sometimes get this error:

panoptes_client.panoptes.PanoptesAPIException: Attempted to touch a stale object: SubjectSet.

Relevant code snippet is:

lNewSubjects = []
        for det in dManifest:
            s=Subject()
            s.links.project=project.id
            s.links.subject_set=subject_set.id
            for filename in dManifest[det]['sImageFilenames']:
                pngFilename = var.sWorkingDirectory + filename.rstrip('.arch.fz') + '_' + det + '.png'
                #Catch for any subject with less than max number of files
                if os.path.isfile(pngFilename):
                    s.add_location(pngFilename)
            if var.bUpload:
                s.save()
                dManifest[det]['iSubjectId'] = int(s.id)
                s.metadata.update(dManifest[det])
                s.save()
                lNewSubjects.append(s)
                #subject_set.add(s)
            else:
                dManifest[det]['iSubjectId'] = None
                
        if var.bUpload:
            try:
                subject_set.add(lNewSubjects)
            except Exception as ex: 
                var.logger.error('SUBJECTUPLOAD Error: ' + str(ex))
                var.logger.error(traceback.format_exc())

Is it not possible to add subjects to a subject set from simultaneous processes?

Thanks for any info!

@camallen
Copy link
Contributor

We use conditional requests to ensure a client doesn't clobber another clients changes.

As such you'll need to reload the subject set to get a refreshed view with the updated changes (most likely new linked subjects count or similar) before trying to add the new batch of subjects.

Running these requests concurrently creates a race condition of state changes, you may have to add an exception handler with some retry logic to correctly handle this issue as it could be unpredictable.

You may want to investigate how we do this in the cli tool https://github.com/zooniverse/panoptes-cli/blob/fb5da0d61fe50d441baef5d62079d1f91e2b5a46/panoptes_cli/commands/subject_set.py#L411-L424 using async_saves() and small batch links to do concurrent uploads that avoids the race condition on the subject set.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants