Skip to content

Add JSON functionality to SQLAlchemy Dialect #194

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
laserkaplan opened this issue Jul 6, 2022 · 2 comments
Closed

Add JSON functionality to SQLAlchemy Dialect #194

laserkaplan opened this issue Jul 6, 2022 · 2 comments

Comments

@laserkaplan
Copy link
Member

In messing around with getting the Trino SQLAlchemy Dialect to work with JSON columns/data, I came across at least a couple of necessary additions:

  • a visit_JSON method is needed in the TrinoTypeCompiler class since SQLAlchemy, by default, doesn't support a JSON type (as it is dialect-specific).
  • the TrinoDialect class needs both a _json_serializer and _json_deserializer member, which I believe can simply be json.dumps and json.loads, respectively.

Adding those two items allowed me to create tables with a JSON type and write to them all using SQLAlchemy. However, when attempting to read back from the table, I got the following error:

Traceback (most recent call last):
  File "/Users/laserkaplan/PycharmProjects/proxima/proxima_warehouse/pipelines/process_shopify_orders/process.py", line 95, in <module>
    print(session.execute(select(sis_table).limit(10)).all())
  File "/Users/laserkaplan/.pyenv/versions/proxima-3.9.7/lib/python3.9/site-packages/sqlalchemy/engine/result.py", line 1068, in all
    return self._allrows()
  File "/Users/laserkaplan/.pyenv/versions/proxima-3.9.7/lib/python3.9/site-packages/sqlalchemy/engine/result.py", line 403, in _allrows
    made_rows = [make_row(row) for row in rows]
  File "/Users/laserkaplan/.pyenv/versions/proxima-3.9.7/lib/python3.9/site-packages/sqlalchemy/engine/result.py", line 403, in <listcomp>
    made_rows = [make_row(row) for row in rows]
  File "/Users/laserkaplan/.pyenv/versions/proxima-3.9.7/lib/python3.9/site-packages/sqlalchemy/sql/sqltypes.py", line 2693, in process
    return json_deserializer(value)
TypeError: loads() takes 1 positional argument but 2 were given

So something is off with the deserializer, I just haven't had the chance to dig into it further yet.

Does anyone have any suggestions of how to proceed here? Or, if you have more SQLAlchemy JSON experience than I do (which isn't hard to do :) ), do you know if anything else is missing?

@mdesmet
Copy link
Contributor

mdesmet commented Jul 13, 2022

Hi @laserkaplan, can you make a PR? This way It will be much easier to have a look at your code and get it working.

@hashhar
Copy link
Member

hashhar commented Sep 22, 2022

Fixed by #214.

@hashhar hashhar closed this as completed Sep 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

3 participants