This commit is contained in:
2025-03-13 12:11:55 +02:00
commit ad530b133f
15 changed files with 240 additions and 0 deletions

BIN
.DS_Store vendored Normal file

Binary file not shown.

5
.gitignore vendored Normal file
View File

@@ -0,0 +1,5 @@
target/
dbt_packages/
logs/
env/

7
.vscode/extensions.json vendored Normal file
View File

@@ -0,0 +1,7 @@
{
"recommendations": [
"innoverio.vscode-dbt-power-user",
"mhutchie.git-graph",
"ms-python.python"
]
}

113
README.md Normal file
View File

@@ -0,0 +1,113 @@
Welcome to your new dbt project!
# Installation
The following tutorial assumes you're already familiar with git and command line usage.
## Getting the code to your local machine
1. Fork this github repository into your local account
2. Copy it to your local machine: `git clone https://github.com/your_account_name/econ250_2025.git`
## gcloud authentication
To run queries from your command line, you'll first need to install `gcloud` utility.
Follow the instructions here: https://cloud.google.com/sdk/docs/install. After installation you should have `gcloud` command available for running in the terminal.
Now, try to authenticate with your **kse email** using the following command:
```bash
gcloud auth application-default login \
--scopes=https://www.googleapis.com/auth/bigquery,\
https://www.googleapis.com/auth/drive.readonly,\
https://www.googleapis.com/auth/iam.test
```
Now, when you run the following commands something similar should be response:
```bash
$ gcloud auth list
Credentialed Accounts
ACTIVE ACCOUNT
* o_omelchenko@kse.org.ua
```
To set the active project, run the following:
```bash
gcloud config set project econ250-2025
```
## venv and libraries
Prerequisites: having Python installed on your machine.
Following instructions are for Linux or WSL; if you'd like to run Windows - please refer to the documentation below.
```bash
# change directory to the one you just copied from github
cd econ250_2025
# create and activate venv
python3 -m venv env
source env/bin/activate
pip install -r requirements.txt
```
If everything is installed correctly, you should run the following commands successfully:
```
$ dbt --version
Core:
- installed: 1.9.3
- latest: 1.9.3 - Up to date!
Plugins:
- bigquery: 1.9.1 - Up to date!
```
For more detailed reference, refer to the official documentation here:
- https://docs.getdbt.com/docs/core/pip-install
- https://docs.getdbt.com/docs/core/connect-data-platform/bigquery-setup#local-oauth-gcloud-setup
## Adjusting the configuration
You'll need to specify your own dataset to save your models to. To do so, navigate to the `profiles.yml` in the root directory of the project, and replace `o_omelchenko` with your bigquery dataset name with which you have been working previously.
## Final check
Try running the following command:
- dbt run
If everything is set up well, you will see similar output:
```log
dbt run
01:18:56 Running with dbt=1.9.3
01:18:57 Registered adapter: bigquery=1.9.1
01:18:57 Found 2 models, 4 data tests, 491 macros
01:18:57
01:18:57 Concurrency: 2 threads (target='dev')
01:18:57
01:19:00 1 of 2 START sql table model o_omelchenko.my_first_dbt_model ................... [RUN]
01:19:04 1 of 2 OK created sql table model o_omelchenko.my_first_dbt_model .............. [CREATE TABLE (2.0 rows, 0 processed) in 4.44s]
01:19:04 2 of 2 START sql view model o_omelchenko.my_second_dbt_model ................... [RUN]
01:19:06 2 of 2 OK created sql view model o_omelchenko.my_second_dbt_model .............. [CREATE VIEW (0 processed) in 2.13s]
01:19:06
01:19:06 Finished running 1 table model, 1 view model in 0 hours 0 minutes and 9.64 seconds (9.64s).
```
If you have any troubles with installation, please contact the course instructor (Oleh Omelchenko) in slack for assist.

0
analyses/.gitkeep Normal file
View File

36
dbt_project.yml Normal file
View File

@@ -0,0 +1,36 @@
# Name your project! Project names should contain only lowercase characters
# and underscores. A good package name should reflect your organization's
# name or the intended use of these models
name: 'econ250_2025'
version: '1.0.0'
# This setting configures which "profile" dbt uses for this project.
profile: 'econ250_2025'
# These configurations specify where dbt should look for different types of files.
# The `model-paths` config, for example, states that models in this project can be
# found in the "models/" directory. You probably won't need to change these!
model-paths: ["models"]
analysis-paths: ["analyses"]
test-paths: ["tests"]
seed-paths: ["seeds"]
macro-paths: ["macros"]
snapshot-paths: ["snapshots"]
clean-targets: # directories to be removed by `dbt clean`
- "target"
- "dbt_packages"
# Configuring models
# Full documentation: https://docs.getdbt.com/docs/configuring-models
# In this example config, we tell dbt to build all models in the example/
# directory as views. These settings can be overridden in the individual model
# files using the `{{ config(...) }}` macro.
models:
econ250_2025:
# Config indicated by + and applies to all files under models/example/
example:
+materialized: view

0
macros/.gitkeep Normal file
View File

View File

@@ -0,0 +1,27 @@
/*
Welcome to your first dbt model!
Did you know that you can also configure models directly within SQL files?
This will override configurations stated in dbt_project.yml
Try changing "table" to "view" below
*/
{{ config(materialized='table') }}
with source_data as (
select 1 as id
union all
select null as id
)
select *
from source_data
/*
Uncomment the line below to remove records with null `id` values
*/
-- where id is not null

View File

@@ -0,0 +1,6 @@
-- Use the `ref` function to select from other models
select *
from {{ ref('my_first_dbt_model') }}
where id = 1

21
models/example/schema.yml Normal file
View File

@@ -0,0 +1,21 @@
version: 2
models:
- name: my_first_dbt_model
description: "A starter dbt model"
columns:
- name: id
description: "The primary key for this table"
data_tests:
- unique
- not_null
- name: my_second_dbt_model
description: "A starter dbt model"
columns:
- name: id
description: "The primary key for this table"
data_tests:
- unique
- not_null

22
profiles.yml Normal file
View File

@@ -0,0 +1,22 @@
econ250_2025:
outputs:
dev:
dataset: o_omelchenko # TODO: Change this to your personal dataset
job_execution_timeout_seconds: 300
job_retries: 1
location: US
method: oauth
priority: interactive
project: econ250-2025
threads: 2
type: bigquery
target: dev
my-bigquery-db:
outputs:
dev:
dataset: o_omelchenko # TODO: Change this to your personal dataset
method: oauth
project: econ250-2025
threads: 4
type: bigquery
target: dev

3
requirements.txt Normal file
View File

@@ -0,0 +1,3 @@
dbt-bigquery==1.9.1
dbt-core==1.9.3
google-cloud-core==2.4.3

0
seeds/.gitkeep Normal file
View File

0
snapshots/.gitkeep Normal file
View File

0
tests/.gitkeep Normal file
View File