Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ingest non-2MASS JHK photometry from Ultracool sheet #526

Open
kelle opened this issue Jul 1, 2024 · 6 comments · May be fixed by #552
Open

Ingest non-2MASS JHK photometry from Ultracool sheet #526

kelle opened this issue Jul 1, 2024 · 6 comments · May be fixed by #552
Assignees

Comments

@kelle
Copy link
Collaborator

kelle commented Jul 1, 2024

Part of #452

@kilohoku - I see some JHK measurements in the "2MASS" columns which don't appear to be 2MASS sources. Are the majority of these from a particular instrument/telescope that is not 2MASS?

@kelle kelle mentioned this issue Jul 1, 2024
10 tasks
@kilohoku
Copy link

kilohoku commented Jul 2, 2024 via email

@kelle
Copy link
Collaborator Author

kelle commented Jul 8, 2024

Given that we don't ingest synthetic or calculated photometry, i'm taking this action item off the list. Thanks for the quick response!

@kelle kelle closed this as not planned Won't fix, can't repro, duplicate, stale Jul 8, 2024
@kilohoku
Copy link

kilohoku commented Jul 10, 2024 via email

@kelle kelle reopened this Jul 11, 2024
@Exu-112
Copy link
Collaborator

Exu-112 commented Aug 18, 2024

It seems like the update to astrodb_utils a few days ago broke the find_source_in_db function. Im getting this error:

Traceback (most recent call last): File "/Users/florence/Astro/SIMPLE-db/scripts/ingests/ultracool_sheet/Ingest_VISTA_K_photometry.py", line 38, in <module> db = load_astrodb( File "/opt/anaconda3/envs/simple-db/lib/python3.10/site-packages/astrodb_utils/utils.py", line 95, in load_astrodb db.load_database( File "/opt/anaconda3/envs/simple-db/lib/python3.10/site-packages/astrodbkit2/astrodb.py", line 976, in load_database self.load_json(os.path.join(directory_of_sources, file)) File "/opt/anaconda3/envs/simple-db/lib/python3.10/site-packages/astrodbkit2/astrodb.py", line 909, in load_json conn.execute(self.metadata.tables[self._primary_table].insert().values(data[self._primary_table])) File "/opt/anaconda3/envs/simple-db/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1414, in execute return meth( File "/opt/anaconda3/envs/simple-db/lib/python3.10/site-packages/sqlalchemy/sql/elements.py", line 485, in _execute_on_connection return connection._execute_clauseelement( File "/opt/anaconda3/envs/simple-db/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1630, in _execute_clauseelement compiled_sql, extracted_params, cache_hit = elem._compile_w_cache( File "/opt/anaconda3/envs/simple-db/lib/python3.10/site-packages/sqlalchemy/sql/elements.py", line 664, in _compile_w_cache compiled_sql = self._compiler( File "/opt/anaconda3/envs/simple-db/lib/python3.10/site-packages/sqlalchemy/sql/elements.py", line 290, in _compiler return dialect.statement_compiler(dialect, self, **kw) File "/opt/anaconda3/envs/simple-db/lib/python3.10/site-packages/sqlalchemy/sql/compiler.py", line 1269, in __init__ Compiled.__init__(self, dialect, statement, **kwargs) File "/opt/anaconda3/envs/simple-db/lib/python3.10/site-packages/sqlalchemy/sql/compiler.py", line 710, in __init__ self.string = self.process(self.statement, **compile_kwargs) File "/opt/anaconda3/envs/simple-db/lib/python3.10/site-packages/sqlalchemy/sql/compiler.py", line 755, in process return obj._compiler_dispatch(self, **kwargs) File "/opt/anaconda3/envs/simple-db/lib/python3.10/site-packages/sqlalchemy/sql/visitors.py", line 143, in _compiler_dispatch return meth(self, **kw) # type: ignore # noqa: E501 File "/opt/anaconda3/envs/simple-db/lib/python3.10/site-packages/sqlalchemy/sql/compiler.py", line 5317, in visit_insert crud_params_struct = crud._get_crud_params( File "/opt/anaconda3/envs/simple-db/lib/python3.10/site-packages/sqlalchemy/sql/crud.py", line 326, in _get_crud_params raise exc.CompileError( sqlalchemy.exc.CompileError: Unconsumed column names: dec, ra

It seems to relate to the change from ra and dec to ra_deg and dec_deg in the column names. I tried changing schema to reflect this, but I couldn't load the database then as the column names were different. The data files would need to be updated somehow if this change to schema is to be implemented.

@Exu-112
Copy link
Collaborator

Exu-112 commented Aug 18, 2024

Running pytest reveals nothing wrong. Perhaps more tests are needed to check integration with astrodb_utils?

@kelle
Copy link
Collaborator Author

kelle commented Aug 19, 2024

It looks like this error is actually in load_astrodb not find_source_in_db. One thing that has changed since you last ran this script is the change in the way the JSON files are organized. I tagged you in a comment on Slack that could help.

@Exu-112 Exu-112 linked a pull request Aug 21, 2024 that will close this issue
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants