Skip to content

Commit d9c1e22

Browse files
koppkopp
committed
Factor development setup into DEVELOPMENT.md
1 parent f05b6e3 commit d9c1e22

File tree

5 files changed

+217
-156
lines changed

5 files changed

+217
-156
lines changed

‎bindings/kepler.gl-jupyter/DEVELOPMENT.md

Lines changed: 205 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,205 @@
1+
# Local Development
2+
3+
## Preface
4+
5+
This file describes the keplergl bindings for jupyter, i.e. the python package you can install as `keplergl` (see below).
6+
7+
It makes use of the `kepler.gl` javascript package, which is defined in `../..`, but pulls this dependency from [upstream](https://www.npmjs.com/package/kepler.gl).
8+
Hence to build this binding it is _not_ required to build the parent project.
9+
10+
### Provided entities
11+
12+
The bindings actually provide two things:
13+
14+
- A JupyterLab/Jupyter Notebook extension, which is displayed/interacted with in your browser when using `jupyter`.
15+
- A python package, that can be interacted with via `import keplergl` in python.
16+
17+
The python package bundles all of these together.
18+
19+
20+
### Code organization
21+
22+
The python package is declared by `./pyproject.toml`.
23+
The actual python code is in `./keplergl`.
24+
This provides a jupyter widget (a `ipywidgets.DOMWidget`).
25+
26+
This python package contains and makes use of the javascript package `keplergl-jupyter`, which resides in [`./js`](./js).
27+
It provides a [JupyterLab extension](https://jupyterlab.readthedocs.io/en/stable/extension/extension_dev.html) (see `./js/labplugin.js`) and a [Jupyter Notebook extension](https://nbclassic.readthedocs.io/en/latest/extending/index.html) (see `./js/extension.js`) which share all the 'functional' code to display the kepler.gl-widget.
28+
29+
When building the js code, this will generate javascript files in `./keplergl/static`.
30+
When installing the python package, the code in `./js` is also automatically built (unless the build results are already there, i.e. if you manually built it).
31+
32+
Some example notebooks can be found in `./notebooks`.
33+
34+
35+
### Release
36+
37+
See [`RELEASE.md`](RELEASE.md) for how this package is released.
38+
39+
40+
## Setup
41+
42+
### Environment Setup
43+
You will need to install node, yarn and Jupyter Notebook.
44+
45+
#### 1. Node and Yarn
46+
We recommended to use [`nvm`](https://github.com/nvm-sh/nvm) to install `node` and to use `yarn` via `corepack`:
47+
48+
cd js
49+
nvm install # this will install `node` in the version specified in `.nvmrc`
50+
corepack enable
51+
yarn # this will install yarn in the version specified in `yarn.lock`.
52+
53+
It may work to use other [node](https://nodejs.org/en/download/package-manager) `> 12` and [yarn](https://yarnpkg.com/getting-started/install) installations.
54+
55+
#### Jupyter
56+
57+
58+
- Using conda
59+
```shell
60+
conda install jupyter
61+
conda install notebook 6.0.1
62+
```
63+
64+
- Using pip
65+
(When working with `pip` you may want to first create a virtual environment via `python -m venv venv && source ./venv/bin/activate` -- on Ubuntu you will need to install `python3-venv` for that)
66+
```shell
67+
pip install jupyter
68+
pip install notebook==6.0.1
69+
```
70+
71+
#### 3. Install GeoPandas
72+
73+
- Using conda
74+
```shell
75+
conda install geopandas
76+
```
77+
78+
- Using pip
79+
80+
```shell
81+
pip install geopandas
82+
```
83+
84+
### Download and run keplergl in your local Jupyter Notebook
85+
86+
#### Clone Repo
87+
```shell
88+
git clone https://github.com/keplergl/kepler.gl.git
89+
```
90+
91+
### Setup JS
92+
#### 1. Install Js dependencies
93+
```sh
94+
cd bindings/kepler.gl-jupyter
95+
cd js
96+
yarn
97+
```
98+
99+
Rerun `yarn` if any js dependency has changed (usually after pulling new changes from master).
100+
101+
#### 2. Load mapbox token
102+
Add a [Mapbox access token](https://docs.mapbox.com/help/how-mapbox-works/access-tokens/) to the environment.
103+
104+
```sh
105+
export MapboxAccessTokenJupyter=<your_mapbox_token>
106+
```
107+
108+
#### 3. Build the js module
109+
110+
For development of the js code, it is convenient to start a local server to watch for changes:
111+
112+
```shell
113+
yarn start
114+
```
115+
116+
If you edit a file in the `js` directory, the local server will pick up those changes.
117+
118+
To build without a local server that watches for changes, use
119+
120+
```shell
121+
yarn build
122+
```
123+
124+
### Setup jupyter
125+
126+
#### 1. Install python module and enable extension from local files
127+
This command must be run **AFTER** the `js` setup, and folder `./keplergl/static/` was created. It only needs to be run once.
128+
129+
```sh
130+
# dev install from folder containing setup.py
131+
pip install -e .
132+
133+
# only needed in dev mode, not in normal mode.
134+
jupyter nbextension install --py --symlink --sys-prefix keplergl
135+
136+
# only needed in dev mode, not in normal mode.
137+
jupyter nbextension enable --py --sys-prefix keplergl
138+
```
139+
140+
NOTE:
141+
The above command `jupyter nbextension install -py --symlink --sys-prefix keplergl` is trying to create a symoblic link from the folder `bindings/kepler.gl-jupyter/keplergl/static` into jupyter's folder `nbextensions`.
142+
Please check if there folder `nbextensions/kepler-jupyter` already exists and if so, remove (or rename) it first.
143+
144+
To find the location of `nbextensions` folder, you can use the following command:
145+
```shell
146+
realpath $(dirname $(dirname $(which jupyter)))/share/jupyter/nbextensions
147+
```
148+
For example for
149+
```
150+
$ where jupyter
151+
/Users/test/opt/anaconda3/envs/test37/bin/jupyter
152+
```
153+
the nbextensions should be at:
154+
```
155+
/Users/test/opt/anaconda3/envs/test37/share/jupyter/nbextensions
156+
```
157+
158+
159+
#### 2. Start jupyter notebook
160+
161+
```shell
162+
cd notebooks
163+
jupyter notebook
164+
```
165+
166+
167+
#### Have fun!
168+
169+
You can now start editing the `.js` and `.py` files to see changes reflected in your local notebook. After changing files in the `./js` folder, the local start script will recompile the `.js` files and put them in to `keplergl/static` folder. You need to reload the jupyter notebook page to reload the files.
170+
171+
172+
#### 3. Development for JupyterLab
173+
174+
To target JupyterLab instead of Jupyter Notebook, use
175+
176+
```shell
177+
pip install -e . # in this folder
178+
cd js && jupyter labextension develop . --overwrite
179+
cd ../notebooks && jupyter lab
180+
```
181+
for development.
182+
This will automatically create the correct symlinks to let `jupyter lab` pick up the extension.
183+
184+
To build and install it use (in `./js`)
185+
```shell
186+
jupyter labextension build .
187+
jupyter labextension install .
188+
```
189+
190+
Alternatively you can build the extension via `jupyter labextension build .` and then manually create a symbolic link from the folder output folder `./kepler-jupyter/labextension` to JupyterLab's folder `labextensions`, e.g. `/Users/test/opt/anaconda3/envs/test37/share/jupyter/labextensions`. You will need to reload the jupyter lab page.
191+
192+
The output of the jupyter labextension is defined in the file `js/package.json`:
193+
```javascript
194+
...
195+
"jupyterlab": {
196+
"extension": "babel/labplugin",
197+
"outputDir": "../keplergl-jupyter/labextension",
198+
"sharedPackages": {
199+
"@jupyter-widgets/base": {
200+
"bundled": false,
201+
"singleton": true
202+
}
203+
}
204+
}
205+
```

‎bindings/kepler.gl-jupyter/README.md

Lines changed: 7 additions & 155 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,15 @@
11
# kepler.gl for Jupyter
22

3-
This is the [kepler.gl](http://kepler.gl) jupyter widget, an advanced geospatial visualization tool, to render large-scale interactive maps in Jupyter Notebook.
3+
This is the [kepler.gl](http://kepler.gl) jupyter widget, an advanced geospatial visualization tool, to render large-scale interactive maps in Jupyter Notebook and JupyterLab.
44

5-
![Kepler.gl for Jupyter][jupyter_widget]
5+
![Kepler.gl for Jupyter](https://d1a3f4spazzrp4.cloudfront.net/kepler.gl/documentation/jupyter_widget.png)
66

7-
Table of contacts
7+
Table of contents
88
- [Installation](#installation)
99
- [Quick Start](#quick-start)
1010
- [Demo Notebooks](#demo-notebooks)
1111
- [Usage](#usage)
1212
- [Local Development Setup](#local-development-setup)
13-
- [FAQ & Troubleshoot](#faq--troubleshoot)
1413

1514
## Installation
1615

@@ -199,8 +198,10 @@ Please note that the map is not interactive due to the limitation of Google Cola
199198
- [GeoDataFrame](https://github.com/keplergl/kepler.gl/blob/master/bindings/kepler.gl-jupyter/notebooks/GeoDataFrame.ipynb): Load GeoDataFrame to kepler.gl
200199

201200

202-
https://docs.kepler.gl/docs/keplergl-jupyter#1-load-keplergl-map
203201
## Usage
202+
203+
See documentation at https://docs.kepler.gl/docs/keplergl-jupyter
204+
204205
- [1. Load kepler.gl](https://docs.kepler.gl/docs/keplergl-jupyter#1-load-keplergl-map)
205206
- [`keplergl.KeplerGl()`](https://docs.kepler.gl/docs/keplergl-jupyter#keplergl)
206207
- [2. Add Data](https://docs.kepler.gl/docs/keplergl-jupyter#2-add-data)
@@ -221,155 +222,6 @@ https://docs.kepler.gl/docs/keplergl-jupyter#1-load-keplergl-map
221222

222223
## Local Development Setup
223224

224-
### Environment Setup
225-
You will need to install node, yarn and Jupyter Notebook.
226-
227-
#### 1. Node and Yarn
228-
Install [node](https://nodejs.org/en/download/package-manager/#macos) `> 12`, and [yarn](https://yarnpkg.com/en/docs/install#mac-stable). Use [nvm](https://github.com/creationix/nvm) for better node version management e.g. `nvm install 12`.
229-
230-
231-
#### 2. Install Jupyter
232-
233-
- Using conda
234-
```shell
235-
conda install jupyter
236-
conda install notebook 6.0.1
237-
```
238-
239-
- Using pip
240-
241-
```shell
242-
pip install jupyter
243-
pip install notebook==6.0.1
244-
```
245-
246-
#### 3. Install GeoPandas
247-
248-
- Using conda
249-
```shell
250-
conda install geopandas
251-
```
252-
253-
- Using pip
254-
255-
```shell
256-
pip install geopandas
257-
```
258-
259-
### Download and run keplergl in your local Jupyter Notebook
260-
261-
#### Clone Repo
262-
```shell
263-
git clone https://github.com/keplergl/kepler.gl.git
264-
```
265-
266-
### Setup JS
267-
#### 1. Install Js module
268-
```sh
269-
cd bindings/kepler.gl-jupyter
270-
cd js
271-
yarn
272-
```
273-
274-
#### 2. Load mapbox token
275-
Add [Mapbox access token](https://docs.mapbox.com/help/how-mapbox-works/access-tokens/) to Node env.
276-
277-
```sh
278-
export MapboxAccessTokenJupyter=<your_mapbox_token>
279-
```
280-
281-
#### 3. Build js module, start a local server to watch for changes
282-
283-
```shell
284-
npm start
285-
```
286-
287-
You need to run step 2 and 3 to restart the js program. And step 1-3 if any js dependency has changed (Usually after pulling new changes from master).
288-
289-
### Setup jupyter
290-
291-
#### 1. Install python module and enable extension from local files
292-
This command must be run **AFTER** the `js` setup, and folder `static/` was created. It only needs to be run once.
293-
294-
```sh
295-
# dev install from folder containing setup.py
296-
pip install -e .
297-
298-
# only needed in dev mode, not in normal mode.
299-
jupyter nbextension install --py --symlink --sys-prefix keplergl
300-
301-
# only needed in dev mode, not in normal mode.
302-
jupyter nbextension enable --py --sys-prefix keplergl
303-
```
304-
305-
NOTE: The above command `jupyter nbextension install -py --symlink --sys-prefix keplergl` is trying to create a symoblic link of the folder `bindings/kepler.gl-jupyter/keplergl/static` under the jupyter's folder `nbextensions`. Please check if there is already a folder "nbextensions/kepler-jupyter" existed, and you might need to remove it first.
306-
307-
To find the location of `nbextensions` folder, you can use the following command:
308-
```shell
309-
$ where jupyter
310-
/Users/test/opt/anaconda3/envs/test37/bin/jupyter
311-
312-
# the nbextensions should be at: /Users/test/opt/anaconda3/envs/test37/share/jupyter/nbextensions
313-
```
225+
See file [`DEVELOPMENT.md`](DEVELOPMENT.md)
314226

315227

316-
#### 2. Start jupyter notebook
317-
318-
```shell
319-
cd notebooks
320-
jupyter notebook
321-
```
322-
323-
324-
#### Have fun!
325-
326-
You can now start editing the .js and .py files to see changes reflected in your local notebook. After changing files in the js folder, the local start script will recompile the js files and put them in to `keplergl/static` folder. You need to reload the jupyter notebook page to reload the files.
327-
328-
329-
#### 3. Development for JupyterLab
330-
331-
To test the development work in previous 2 steps for JupyterLab. You can build the `keplergl labextension` under the `js` directory:
332-
333-
```shell
334-
jupyter labextension build .
335-
```
336-
337-
The output of the jupyter labextension is defined in the file `js/package.json`:
338-
```javascript
339-
...
340-
"jupyterlab": {
341-
"extension": "babel/labplugin",
342-
"outputDir": "../keplergl-jupyter/labextension",
343-
"sharedPackages": {
344-
"@jupyter-widgets/base": {
345-
"bundled": false,
346-
"singleton": true
347-
}
348-
}
349-
}
350-
```
351-
352-
Then, you can either install this labextension to test it:
353-
```shell
354-
jupyter labextension install .
355-
```
356-
or, you can manually create a symbolic link for the folder `bindings/kepler.gl-jupyter/kepler-jupyter/labextension` under the jupyter's folder `labextensions`, e.g. `/Users/test/opt/anaconda3/envs/test37/share/jupyter/labextensions`. You will need to reload the jupyter lab page.
357-
358-
[jupyter_widget]: https://d1a3f4spazzrp4.cloudfront.net/kepler.gl/documentation/jupyter_widget.png
359-
[empty_map]: https://d1a3f4spazzrp4.cloudfront.net/kepler.gl/documentation/jupyter_empty_map.png
360-
[geodataframe_map]: https://d1a3f4spazzrp4.cloudfront.net/kepler.gl/documentation/jupyter_geodataframe.png
361-
[map_interaction]: https://d1a3f4spazzrp4.cloudfront.net/kepler.gl/documentation/jupyter_custom_map.gif
362-
[load_map_w_data]: https://d1a3f4spazzrp4.cloudfront.net/kepler.gl/documentation/jupyter_load_map_w_data.gif
363-
[map_add_data]: https://d1a3f4spazzrp4.cloudfront.net/kepler.gl/documentation/jupyter_add_data.png
364-
[connect_data_config]: https://d1a3f4spazzrp4.cloudfront.net/kepler.gl/documentation/jupyter_connect_data_w_config.png
365-
[save_widget_state]: https://d1a3f4spazzrp4.cloudfront.net/kepler.gl/documentation/jupyter_save_state.png
366-
367-
[wkt]: https://dev.mysql.com/doc/refman/5.7/en/gis-data-formats.html#gis-wkt-format
368-
[geojson]: https://tools.ietf.org/html/rfc7946
369-
[feature_collection]: https://tools.ietf.org/html/rfc7946#section-3.3
370-
[features]: https://tools.ietf.org/html/rfc7946#section-3.2
371-
[data_frame]: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html
372-
[geo_data_frame]: https://geopandas.readthedocs.io/en/latest/data_structures.html#geodataframe
373-
374-
[match-config-w-data]: #match-config-with-data
375-
[data_format]: #3-data-format

‎bindings/kepler.gl-jupyter/js/.nvmrc

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
../../../.nvmrc

0 commit comments

Comments
 (0)