Configuration¶
YAML Configuration File¶
application_name - the name of the application
connections - a list of connections used in the application. You may as many connections as you require.
async_mode - a boolean value that determines if the application should run in async mode. Default is false
Connections are further defined by:
-
name - name of the connection that is used to refer to and identify this connection
-
backend - a python path to connection implementation class
-
credentials - credentials required by the connection that will be passed to the
connect
method of your connection implementation class.
resources_config - the top level resource allocation of the apuplication, including:
-
lakehouse - the name of connection (specified in connections section above) that will be used for the AMSDAL Lakehouse connection
-
lock - the name of connection that will be used for locking functionality required to keep your data consistent
-
repository - connection that should be used by default as well as for each model
- default: connection name
- models:
- model_name_a: connection name
- model_name_b: connection name
- ...
Example¶
application_name: MyApp
connections:
- name: lakehouse-connection-1
backend: postgres-historical
credentials:
configs:
spark.ui.enabled: false
- name: data-connection-2
backend: sqlite
credentials:
- db_path: ./default.db
- name: data-connection-3
backend: sqlite
credentials:
- db_path: ./users.db
- name: lock-connection
backend: amsdal_data.lock.implementations.redis_lock.RedisLock
credentials:
- host: localhost
- port: 6379
resources_config:
lakehouse: lakehouse-connection-1
lock: lock-connection
repository:
default: data-connection-2
models:
user: data-connection-3
application_name: MyApp
async_mode: true
connections:
- name: lakehouse-connection-1
backend: postgres-historical-async
credentials:
configs:
spark.ui.enabled: false
- name: data-connection-2
backend: sqlite-async
credentials:
- db_path: ./default.db
- name: data-connection-3
backend: sqlite-async
credentials:
- db_path: ./users.db
- name: lock-connection
backend: amsdal_data.lock.implementations.redis_lock.RedisLock
credentials:
- host: localhost
- port: 6379
resources_config:
lakehouse: lakehouse-connection-1
lock: lock-connection
repository:
default: data-connection-2
models:
user: data-connection-3
Review¶
{
"config_path": "./config.yml",
"http_port": 8080,
"check_model_exists": true,
"json_indent": 4
}