Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Possible breaking change in calling post_model_hook in data and unit tests #10822

Open
2 tasks done
edgarrmondragon opened this issue Oct 4, 2024 · 1 comment · May be fixed by #10809
Open
2 tasks done

[Bug] Possible breaking change in calling post_model_hook in data and unit tests #10822

edgarrmondragon opened this issue Oct 4, 2024 · 1 comment · May be fixed by #10809
Labels
awaiting_response bug Something isn't working hooks Issues related to dbt's hooks functionality pre-release Bug not yet in a stable release unit tests Issues related to built-in dbt unit testing functionality

Comments

@edgarrmondragon
Copy link
Contributor

edgarrmondragon commented Oct 4, 2024

Is this a new bug in dbt-core?

  • I believe this is a new bug in dbt-core
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

The following command using the duckdb adapter fails in 1.9.0b1:

dbt --log-level debug build

The data test fail with the following error message

'dict' object has no attribute 'model'

Expected Behavior

The command dbt build should continue working on dbt 1.9

Steps To Reproduce

I'm wrapping the dbt executable with Meltano, but it essentially runs dbt build. The error appeared in this PR: edgarrmondragon/meltano-dogfood#385.

Relevant log output

01:37:52  Running with dbt=1.9.0-b1
01:37:52  running dbt with arguments {'printer_width': '80', 'indirect_selection': 'eager', 'write_json': 'True', 'log_cache_events': 'False', 'partial_parse': 'True', 'cache_selected_only': 'False', 'profiles_dir': '/dev/meltano-dogfood/transform/profiles/duckdb', 'version_check': 'True', 'warn_error': 'None', 'log_path': '/dev/meltano-dogfood/transform/logs', 'debug': 'False', 'fail_fast': 'False', 'use_colors': 'True', 'use_experimental_parser': 'False', 'no_print': 'None', 'quiet': 'False', 'empty': 'False', 'warn_error_options': 'WarnErrorOptions(include=[], exclude=[])', 'invocation_command': 'dbt --log-level debug build', 'introspect': 'True', 'log_format': 'default', 'target_path': '/dev/meltano-dogfood/.meltano/transformers/dbt/target', 'static_parser': 'True', 'send_anonymous_usage_stats': 'False'}
01:37:52  Registered adapter: duckdb=1.8.4
01:37:52  checksum: 342eadeaabeb3b1e51584f606df2b5b2518c39ec1d5d83f52b4bb1c68a4f38ed, vars: {}, profile: , target: , version: 1.9.0b1
01:37:52  Partial parsing enabled: 0 files deleted, 0 files added, 0 files changed.
01:37:52  Partial parsing enabled, no changes found, skipping parsing
01:37:53  Found 4 models, 20 data tests, 4 sources, 2 exposures, 416 macros
01:37:53
01:37:53  Concurrency: 1 threads (target='duckdb')
01:37:53
01:37:53  Acquiring new duckdb connection 'master'
01:37:53  Acquiring new duckdb connection 'list_personal'
01:37:53  Using duckdb connection "list_personal"
01:37:53  On list_personal: /* {"app": "dbt", "dbt_version": "1.9.0b1", "profile_name": "self", "target_name": "duckdb", "connection_name": "list_personal"} */


    select schema_name
    from system.information_schema.schemata

    where catalog_name = '"personal"'



01:37:53  Opening a new connection, currently in state init
Attempting to automatically open the SSO authorization page in your default browser.
1. Please open this link to login into your account: https://auth.motherduck.com/activate
2. Enter the following code: xxxx-xxxx


Token successfully retrieved ✅

You can display the token and store it as an environment variable to avoid having to log in again:
  PRAGMA PRINT_MD_TOKEN;
01:38:20  SQL status: OK in 27.765 seconds
01:38:20  On list_personal: Close
01:38:20  Re-using an available connection from the pool (formerly list_personal, now create_personal_main)
01:38:20  Creating schema "database: "personal"
schema: "main"
"
01:38:20  Using duckdb connection "create_personal_main"
01:38:20  On create_personal_main: /* {"app": "dbt", "dbt_version": "1.9.0b1", "profile_name": "self", "target_name": "duckdb", "connection_name": "create_personal_main"} */


        select type from duckdb_databases()
        where database_name='personal'
        and type='sqlite'


01:38:20  Opening a new connection, currently in state closed
01:38:20  SQL status: OK in 0.001 seconds
01:38:20  Using duckdb connection "create_personal_main"
01:38:20  On create_personal_main: /* {"app": "dbt", "dbt_version": "1.9.0b1", "profile_name": "self", "target_name": "duckdb", "connection_name": "create_personal_main"} */



        create schema if not exists "personal"."main"

01:38:21  SQL status: OK in 0.162 seconds
01:38:21  On create_personal_main: Close
01:38:21  Re-using an available connection from the pool (formerly create_personal_main, now list_personal_main)
01:38:21  Using duckdb connection "list_personal_main"
01:38:21  On list_personal_main: /* {"app": "dbt", "dbt_version": "1.9.0b1", "profile_name": "self", "target_name": "duckdb", "connection_name": "list_personal_main"} */
select
      'personal' as database,
      table_name as name,
      table_schema as schema,
      CASE table_type
        WHEN 'BASE TABLE' THEN 'table'
        WHEN 'VIEW' THEN 'view'
        WHEN 'LOCAL TEMPORARY' THEN 'table'
        END as type
    from system.information_schema.tables
    where table_schema = 'main'
    and table_catalog = 'personal'

01:38:21  Opening a new connection, currently in state closed
01:38:21  SQL status: OK in 0.054 seconds
01:38:21  On list_personal_main: Close
01:38:21  Using duckdb connection "master"
01:38:21  On master: BEGIN
01:38:21  Opening a new connection, currently in state init
01:38:21  SQL status: OK in 0.000 seconds
01:38:21  On master: COMMIT
01:38:21  Using duckdb connection "master"
01:38:21  On master: COMMIT
01:38:21  SQL status: OK in 0.000 seconds
01:38:21  On master: Close
01:38:21  Began running node test.personal_data.source_not_null_github_repositories_id.43e7b57b47
01:38:21  1 of 24 START test source_not_null_github_repositories_id ...................... [RUN]
01:38:21  Re-using an available connection from the pool (formerly list_personal_main, now test.personal_data.source_not_null_github_repositories_id.43e7b57b47)
01:38:21  Began compiling node test.personal_data.source_not_null_github_repositories_id.43e7b57b47
01:38:21  Writing injected SQL for node "test.personal_data.source_not_null_github_repositories_id.43e7b57b47"
01:38:21  Began executing node test.personal_data.source_not_null_github_repositories_id.43e7b57b47
01:38:21  Writing runtime sql for node "test.personal_data.source_not_null_github_repositories_id.43e7b57b47"
01:38:21  Using duckdb connection "test.personal_data.source_not_null_github_repositories_id.43e7b57b47"
01:38:21  On test.personal_data.source_not_null_github_repositories_id.43e7b57b47: /* {"app": "dbt", "dbt_version": "1.9.0b1", "profile_name": "self", "target_name": "duckdb", "node_id": "test.personal_data.source_not_null_github_repositories_id.43e7b57b47"} */
select
      count(*) as failures,
      count(*) != 0 as should_warn,
      count(*) != 0 as should_error
    from (






select id
from "personal"."tap_github"."repositories"
where id is null




    ) dbt_internal_test
01:38:21  Opening a new connection, currently in state closed
01:38:21  SQL status: OK in 0.168 seconds
01:38:21  On test.personal_data.source_not_null_github_repositories_id.43e7b57b47: Close
01:38:21  Unhandled error while executing /dev/meltano-dogfood/.meltano/transformers/dbt/target/run/personal_data/models/github/github.yml/source_not_null_github_repositories_id.sql
'dict' object has no attribute 'model'
01:38:21  Traceback (most recent call last):
  File "/dev/meltano-dogfood/.meltano/utilities/dbt-duckdb/venv/lib/python3.11/site-packages/dbt/task/base.py", line 361, in safe_run
    result = self.compile_and_execute(manifest, ctx)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/dev/meltano-dogfood/.meltano/utilities/dbt-duckdb/venv/lib/python3.11/site-packages/dbt/task/base.py", line 308, in compile_and_execute
    result = self.run(ctx.node, manifest)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/dev/meltano-dogfood/.meltano/utilities/dbt-duckdb/venv/lib/python3.11/site-packages/dbt/task/base.py", line 408, in run
    return self.execute(compiled_node, manifest)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/dev/meltano-dogfood/.meltano/utilities/dbt-duckdb/venv/lib/python3.11/site-packages/dbt/task/test.py", line 291, in execute
    test_result = self.execute_data_test(test, manifest)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/dev/meltano-dogfood/.meltano/utilities/dbt-duckdb/venv/lib/python3.11/site-packages/dbt/task/test.py", line 168, in execute_data_test
    self.adapter.post_model_hook(context, hook_ctx)
  File "/dev/meltano-dogfood/.meltano/utilities/dbt-duckdb/venv/lib/python3.11/site-packages/dbt/adapters/duckdb/impl.py", line 287, in post_model_hook
    self._clean_up_temp_relation_for_incremental(config)
  File "/dev/meltano-dogfood/.meltano/utilities/dbt-duckdb/venv/lib/python3.11/site-packages/dbt/adapters/duckdb/impl.py", line 256, in _clean_up_temp_relation_for_incremental
    if "incremental" == config.model.get_materialization():
                        ^^^^^^^^^^^^
AttributeError: 'dict' object has no attribute 'model'

01:38:21  1 of 24 ERROR source_not_null_github_repositories_id ........................... [ERROR in 0.19s]
01:38:21  Finished running node 

<Continues>

Environment

- OS: macOS 14.6.1
- Python: 3.11
- dbt: 1.9.0b1

Which database adapter are you using with dbt?

No response

Additional Context

@edgarrmondragon edgarrmondragon added bug Something isn't working triage labels Oct 4, 2024
@dbeatty10 dbeatty10 added unit tests Issues related to built-in dbt unit testing functionality hooks Issues related to dbt's hooks functionality labels Oct 4, 2024
@dbeatty10 dbeatty10 linked a pull request Oct 6, 2024 that will close this issue
5 tasks
@dbeatty10
Copy link
Contributor

Thanks for reporting this @edgarrmondragon !

It looks like the conversion here that this might be limited to the dbt-duckdb adapter and fixed in duckdb/dbt-duckdb#451?

If not, can you share relevant files and commands I could use to reproduce this with dbt-core and dbt-postgres?

@dbeatty10 dbeatty10 added awaiting_response pre-release Bug not yet in a stable release and removed triage labels Oct 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
awaiting_response bug Something isn't working hooks Issues related to dbt's hooks functionality pre-release Bug not yet in a stable release unit tests Issues related to built-in dbt unit testing functionality
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants