Pydeck wrapper for use with CARTO and powered by deck.gl.

Index

Installation

Pydeck-carto is available from pip and conda.

It is recommended to always use a virtual environment. to prevent collisions with other libraries installed on the system

Pydeck-carto is a wrapper of Pydeck to use CARTO, so it is a requirement that will be installed automatically.

Via pip

pip install pydeck-carto

Via conda

conda install -c conda-forge pydeck-carto

Using it on Jupyter notebook

In order to use the library in Jupyter notebook (or jupyter lab) requires that pydeck would be properly enabled

Please follow the latest instructions to enable pydeck for Jupyter here

jupyter nbextension install --sys-prefix --symlink --overwrite --py pydeck
jupyter nbextension enable --sys-prefix --py pydeck

To enable pydeck for JupyterLab (on Mac/Unix-like systems):

jupyter labextension install @jupyter-widgets/jupyterlab-manager
DECKGL_SEMVER=`python -c "import pydeck; print(pydeck.frontend_semver.DECKGL_SEMVER)"`
jupyter labextension install @deck.gl/jupyter-widget@$DECKGL_SEMVER

Authentication

Pydeck-carto supports two types of authentication using the carto-auth package:

Authentication using OAuth
Authentication using M2M file

OAuth

Use your CARTO account to authenticate with Python from a notebook (in local or remote), or from a Python script. This is available for any CARTO user.

from carto_auth import CartoAuth

carto_auth = CartoAuth.from_oauth()

The carto_auth object will be used to obtain the CartoLayer credentials.

This method supports the following parameters:

cache_filepath (str, optional): File path where the token is stored. Default “home()/.carto-auth/token_oauth.json”.
use_cache (bool, optional): Whether the stored cached token should be used. Default True.
open_browser (bool, optional): Whether the web browser should be opened to authorize a user. Default True.

M2M (advanced)

Use a file with M2M credentials to automatically login into a CARTO account. It can be uses for ETL processes .This is available for Enterprise CARTO users.

from carto_auth import CartoAuth

carto_auth = CartoAuth.from_m2m("./carto-credentials.json")

This method supports the following parameters:

filepath (str): File path of the CARTO credentials file.
cache_filepath (str, optional): File path where the token is stored. Default “home()/.carto-auth/token_m2m.json”.
use_cache (bool, optional): Whether the stored cached token should be used. Default True.

CARTO credentials file

To generate CARTO Auth tokens in carto-auth you need to create a carto_credentials.json file with the following content:

{
    "api_base_url": "https://gcp-us-east1.api.carto.com",
    "client_id": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
    "client_secret": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
}

To obtain the file’s content, go to the Developers section in the CARTO Dashboard. More information in https://docs.carto.com/carto-user-manual/developers/carto-for-developers/#carto-for-developers.

API Base URL

You can directly copy the API Base URL. It will be different depending on the region where your account is located.

Built applications

Create a new “Machine to Machine” application to generate the Client ID and Client Secret.

Click on “+ Create new”.
Fill in the name and description. The URL is irrelevant in this case, so feel free to use something like https://example.domain.com.
Open the “Advanced Settings” menu.
In “Application Type” select “Machine to Machine”.
Click “Save” and check that your application is listed.

From the list, copy the new Client ID and Client Secret and add them into your credentials file.

CARTO Layer

CartoLayer renders cloud data from any connection (BigQuery, Snowflake, Redshift, Postgres, Databricks). It’s a wrapper over the CartoLayer in deck.gl.

Pydeck-carto is a package outside of pydeck, so calling pydeck_carto.register_carto_layer() is required to register CartoLayer in pydeck.

Example

import pydeck as pdk
from carto_auth import CartoAuth
from pydeck_carto import register_carto_layer, get_layer_credentials
from pydeck_carto.layer import MapType, CartoConnection

# Authentication with CARTO
carto_auth = CartoAuth.from_oauth()

# Register CartoLayer in pydeck
register_carto_layer()

# Render CartoLayer in pydeck
layer = pdk.Layer(
    "CartoLayer",
    data="carto-demo-data.demo_tables.world_airports",
    type_=MapType.TABLE,
    connection=CartoConnection.CARTO_DW,
    credentials=get_layer_credentials(carto_auth),
    get_fill_color=[238, 77, 90],
    point_radius_min_pixels=2.5,
    pickable=True,
)
map_style = pdk.map_styles.ROAD
view_state = pdk.ViewState(latitude=0, longitude=0, zoom=1)
tooltip={"html": "<b>Name:</b> {name}", "style": {"color": "white"}}
pdk.Deck(layer, map_style=map_style, initial_view_state=view_state, tooltip=tooltip)

# Render CartoLayer in pydeck:
# - SQL query from a BigQuery connection
layer = pdk.Layer(
    "CartoLayer",
    data="""
        SELECT a.geom, a.name
        FROM `carto-demo-data.demo_tables.world_airports` AS a,
             `carto-do-public-data.natural_earth.geography_glo_admin0countries_410` AS g
        WHERE g.ADMIN = 'Spain' AND
              ST_INTERSECTS(a.geom, g.geom)
    """,
    type_=MapType.QUERY,
    connection=pdk.types.String("bigquery"),
    credentials=get_layer_credentials(carto_auth),
    get_fill_color=[238, 77, 90],
    point_radius_min_pixels=2.5,
    pickable=True,
)
map_style = pdk.map_styles.ROAD
view_state = pdk.ViewState(latitude=36, longitude=-7.44, zoom=4)
tooltip = {"html": "<b>Name:</b> {name}", "style": {"color": "white"}}
pdk.Deck(layer, map_style=map_style, initial_view_state=view_state, tooltip=tooltip)

Note that show() function is not supported with CARTO layers yet.

Error management

Any data error is displayed instead of the map to provide instant feedback about the input parameters. For example, the user is not authorized, the connection or the column names do not exist, etc.

Properties

data: (str) Either a SQL query or a name of dataset/tileset.
type_: (pydeck.types.String) Type of the input data. It can be either QUERY, TABLE or TILESET. pydeck_carto.layer.MapType().
connection: (pydeck.types.String) Name of the connection registered in the CARTO Workspace. The connection for the CARTO Data Warehouse is already defined as a constant pydeck_carto.layer.CartoConnection().
geo_column: (pydeck.types.String, optional) Name of the geo_column in the CARTO platform. It also support spatial indexes (h3, quadbin) pydeck_carto.layer.GeoColumnType().
credentials: (dict) Defines the app credentials to gather the information from CARTO. It is recommended to use pydeck_carto.get_layer_credentials() to obtain automatically the token from Oauth using the carto-auth package.
aggregation_exp: (pydeck.types.String, optional) Aggregation SQL expression. Only used for spatial index datasets.
aggregation_res_level: (int, optional) Aggregation resolution level. Only used for spatial index datasets, defaults to 6 for quadbins, 4 for h3.

Check the full list of CartoLayer properties.

Reference

get_layer_credentials(carto_auth) → dict

Get the layer credentials object to gather information from carto warehouses.

The return object has the following structure: {"apiVersion": "v3", "apiBaseUrl": "...", "accessToken": "...",}

register_carto_layer(): Add CartoLayer JS bundle to pydeck’s custom libraries.

class MapType

QUERY: alias of query

TABLE: alias of table

TILESET: alias of tileset

class CartoConnection

CARTO_DW: alias of carto_dw

class GeoColumnType

H3: alias of h3

QUADBIN: alias of quadbin

CARTO Styles

CARTO provides data-driven out-of-the-box styling functions for colors. Check the full list of Carto styles in deck.gl.

Example

import pydeck as pdk
from carto_auth import CartoAuth
from pydeck_carto import register_carto_layer, get_layer_credentials
from pydeck_carto.layer import MapType, CartoConnection
from pydeck_carto.styles import color_bins

# Authentication with CARTO
carto_auth = CartoAuth.from_oauth()

# Register CartoLayer in pydeck
register_carto_layer()

# Render CartoLayer in pydeck with color bins style
layer = pdk.Layer(
    "CartoLayer",
    data="SELECT geom, pct_higher_ed FROM `cartobq.public_account.higher_edu_by_county`",
    type_=MapType.QUERY,
    connection=CartoConnection.CARTO_DW,
    credentials=get_layer_credentials(carto_auth),
    get_fill_color=color_bins("pct_higher_ed", [0, 20, 30, 40, 50, 60, 70], "PinkYl"),
    get_line_color=[0, 0, 0, 100],
    line_width_min_pixels=0.5,
    pickable=True,
)
map_style = pdk.map_styles.ROAD
view_state = pdk.ViewState(latitude=38, longitude=-98, zoom=3)
tooltip={"text": "Higher education percentage: {pct_higher_ed} %"}
pdk.Deck(layer, map_style=map_style, initial_view_state=view_state, tooltip=tooltip)

# Render CartoLayer in pydeck with color categories style
layer = pdk.Layer(
    "CartoLayer",
    data="SELECT geom, landuse_type FROM `cartobq.public_account.wburg_parcels`",
    type_=MapType.QUERY,
    connection=CartoConnection.CARTO_DW,
    credentials=get_layer_credentials(carto_auth),
    get_fill_color=color_categories(
        "landuse_type",
        [
            "Multi-Family Walk-Up Buildings",
            "Multi-Family Elevator Buildings",
            "Mixed Residential And Commercial Buildings",
            "Parking Facilities",
            "1 and 2 Family Buildings",
            "Commercial and Office Buildings",
            "Vacant Land",
            "Public Facilities and Institutions",
            "Transportation and Utility",
            "Open Space and Outdoor Recreation",
            "Industrial and Manufacturing",
        ],
        "Bold",
    ),
    get_line_color=[0, 0, 0, 100],
    line_width_min_pixels=0.5,
    pickable=True,
)
map_style = pdk.map_styles.ROAD
view_state = pdk.ViewState(latitude=40.715, longitude=-73.959, zoom=14)
tooltip={
    "html": "<b>Land use type:</b><br>{landuse_type}",
    "style": {"color": "black", "backgroundColor": "#84D2F6"}
}
pdk.Deck(layer, map_style=map_style, initial_view_state=view_state, tooltip=tooltip)

# Render CartoLayer in pydeck with color continuous style
layer = pdk.Layer(
    "CartoLayer",
    data="SELECT geom, value FROM cartobq.public_account.temps",
    type_=MapType.QUERY,
    connection=CartoConnection.CARTO_DW,
    credentials=get_layer_credentials(carto_auth),
    get_fill_color=color_continuous("value", [70, 75, 80, 85, 90, 95, 100], "Peach"),
    point_radius_min_pixels=2.5,
    pickable=True,
)
map_style = pdk.map_styles.ROAD
view_state = pdk.ViewState(latitude=34, longitude=-98, zoom=3)
tooltip={
    "html": "<b>Temperature:</b> {value}°F",
    "style": {"color": "white"}
}
pdk.Deck(layer, map_style=map_style, initial_view_state=view_state, tooltip=tooltip)

Reference

color_bins(attr: str, domain: list, colors: Union[str, list] = 'PurpOr', null_color: list = [204, 204, 204])

Helper function for quickly creating a color bins style.

Data values of each attribute are rounded down to the nearest value in the domain and are then styled with the corresponding color.

Parameters

attr (str) – Attribute or column to symbolize by.
domain (list) – Assign manual class break values.
colors (Union[str, list], optional) – Color assigned to each domain value. - str: A valid named CARTOColors palette. - list: Array of colors in RGBA [ [r, g, b, [a]] ]. Default is PurpOr.
null_color (list, optional) – Color for null values. Default is [204, 204, 204].

color_categories(attr: str, domain: list, colors: Union[str, list] = 'PurpOr', null_color: list = [204, 204, 204], others_color: list = [119, 119, 119])

Helper function for quickly creating a color category style.

Data values of each attribute listed in the domain are mapped one to one with corresponding colors in the range.

Parameters

attr (str) – Attribute or column to symbolize by.
domain (list) – Category list. Must be a valid list of categories.
colors (Union[str, list], optional) – Color assigned to each domain value. - str: A valid named CARTOColors palette. - list: Array of colors in RGBA [ [r, g, b, [a]] ]. Default: PurpOr.
null_color (list, optional) – Color for null values. Default is [204, 204, 204].
others_color (list, optional) – Fallback color for a category not correctly assigned. Default is [119, 119, 119].

color_continuous(attr: str, domain: list, colors: Union[str, list] = 'PurpOr', null_color: list = [204, 204, 204])

Helper function for quickly creating a color continuous style.

Data values of each field are interpolated linearly across values in the domain and are then styled with a blend of the corresponding color in the range.

Parameters

attr (str) – Attribute or column to symbolize by.
domain (list) – Attribute domain to define the data range.
colors (Union[str, list], optional) – Color assigned to each domain value. - str: A valid named CARTOColors palette. - list: Array of colors in RGBA [ [r, g, b, [a]] ]. Default is PurpOr.
null_color (list, optional) – Color for null values. Default is [204, 204, 204].

Contributing

We encourage users to report bugs, fix them, and add features as desired. If you run into issues while using this guide, let us know.

For governance policy and code of conduct, please see the deck.gl contribution guidelines.

Development installation

Please develop using Python 3.7 or above.

git clone https://github.com/visgl/deck.gl
cd deck.gl/bindings/pydeck-carto
make init

At this point, verify that this new local copy works by running make test.

Submitting a pull request

Deck.gl will run a suite of local tests both on commit and on push. On push, deck.gl will run browser tests, which will take a bit longer than the commit hook tests. Ideally, these tests will pass locally before you push your branch to GitHub. Once pushed, tests will also run on Travis CI. Generally the deck.gl team will review your PR within 2-3 days.

Before submitting a PR, you should run make lint and make test to verify that your Python tests pass locally. It may be helpful to run pip install -e . to rebuild pydeck-carto locally.

Building the documentation

To build the documentation locally, run the following:

cd deck.gl/bindings/pydeck-carto/docs
make clean && make html

You can find the homepage at pydeck/docs/_build/html/index.html.

Running python3 -m http.server from pydeck/docs/_build/html will serve the documentation locally.

CHANGELOG

Releases and associated GitHub PRs for pydeck-carto are documented here.

0.1 Releases

0.1.0 - Nov 04 2022

Add register_carto_layer function: integration with pydeck (CartoLayer).
Add get_layer_credentials function: integration with carto-auth.
Add layer enums: MapType, CartoConnection, GeoColumnType.
Add styling functions: color_bins, color_categories, color continuous.
Add implicit on_data_error notifier