Emily CLI Documentation
Release-v3.1.0Emily combines the powers of Python and Docker to build stable and consistent machine learning and datascience python environments. Emily is useful for large cross-team project development as well as for simply running a single jupyter notebook or python script.
Downloademily@latestSee earlier versions
$ emily build [options]Start a new project Valid Emily images: base | slim | cv | nlp torch-slim | torch-cv | torch-nlp | tf-slim tf-cv | tf-nlp | cuda-base | cuda-slim cuda-cv | cuda-nlp |cuda-torch-slim| cuda-torch-cv cuda-torch-nlp| cuda-tf-slim | cuda-tf-cv | cuda-tf-nlp
$ emily build√ Emily: Project name::
· getting-startedEmily: An Emily template is a pre-built template containingessential boilerplate code useful in the development ofmicroservices.These templates include relevant endpoints depending onwhich template you choose.? Emily: Which Emily template do you want to use?(Use arrow keys, confirm with ENTER) … > Default - Minimal project with hello world scriptAPI - Simple project with API set up and readyMachine learning [API, DVC, MLFlow, Pytorch] - Machine learning template with API, Data version control (DVC), MLFlow experiment reporting and pytorchMachine learning API [API, Pytorch] - Machine learning template with API and pytorchMachine learning [gRPC] - Machine learning template with gRPCMachine Learning Reporting - Machine learning reporting template · DefaultEmily: An Emily image is a pre-built docker image containingessential packages that are useful in the development ofmicroservices.These images include relevant packages depending on whichtype of project you are developing.? Emily: Which Emily image fits your use case the best?(Use arrow keys, confirm with ENTER) … > Base - Contains just enough to run an APISlim - Contains essential Machine Learning packagesComputer Vision - Contains slim and essential Computer Vision packagesNLP - Contains slim and essential NLP packagesCustom - Use your own image · Base? Emily: Which editor do you want to use?(Use arrow keys, confirm with ENTER) … Visual Studio Code> PyCharmJupyter NotebookJupyter Lab · PyCharmEmily: Creating a new Emily project in my-directory/my-project-name...$ emily build$ emily build-n my-project-name-i base-t ml-api-e lab$ emily build√ Emily: Project name::
· getting-startedEmily: An Emily template is a pre-built template containing essentialboilerplate code useful in the development of microservices.These templates include relevant endpoints depending onwhich template you choose.? Emily: Which Emily template do you want to use?(Use arrow keys, confirm with ENTER) … Default - Minimal project with hello world script> API - Simple project with API set up and readyMachine learning [API, DVC, MLFlow, Pytorch] - Machine learning template with API, Data version control (DVC), MLFlow experiment reporting and pytorchMachine learning API [API, Pytorch] - Machine learning template with API and pytorchMachine learning [gRPC] - Machine learning template with gRPCMachine Learning Reporting - Machine learning reporting template · APIEmily: An Emily image is a pre-built docker image containingessential packages that are useful in the development of microservices.These images include relevant packages depending on whichtype of project you are developing.? Emily: Which Emily image fits your use case the best?(Use arrow keys, confirm with ENTER) … > Base - Contains just enough to run an APISlim - Contains essential Machine Learning packagesComputer Vision - Contains slim and essential Computer Vision packagesNLP - Contains slim and essential NLP packagesCustom - Use your own image · BaseRunning Emily Build
To build a new Emily project, run emily build in your terminal. You will be asked to provide a project name,
choose a project template (e.g. Default, API, ML-API, etc.), an Emily image
and an editor (e.g. VSCode, PyCharm, Jupyter Notebook or Jupyter Lab).
If you want to build a project without automatically starting a docker container use the flag --no-autostart
When building a project, Emily will let you choose between different project templates and Emily images.
Project templates
A project template is a pre-built selection of software and code snippets that are useful for various use-cases. Emily provides the following project templates:
| Template name | Description |
|---|---|
| Default | Minimal project with hello world script |
| API | Simple project with API set up and ready |
| Machine learning [API, DvC, MLFlow, Pytorch] | Machine learning template with API, data version control (DVC), MLFlow experiment reporting and pytorch |
| Machine learning [API, Pytorch] | Machine learning template with API and pytorch |
| Machine learning [gRPC] | Machine learning template with gRPC |
| Machine learning reporting | Machine learning reporting template |
For the purpose of creating the first project, we will go with the API template. After selecting the template, your terminal output should resemble the following:
Emily images
Emily comes with a selection of pre-built images configured for different usecases:
- Basic tasks that include an API (
base) - Machine learning and data science (
slim) - Computer vision and image processing (
cv) - Natural language processing (
nlp)
Included packages
The Emily images are pre-built with the following packages :
| base | slim | cv | nlp |
|---|---|---|---|
| GCC | base + | slim + | slim + |
| Make | GTK | pkg-config | |
| CMake | + 8 packages1 | libicu-dev | |
| G++ | |||
| Python 3.8 with: | Extra Python packages: | Extra Python packages: | Extra Python packages: |
| - protobuf | - scikit-learn | - OpenCV | - spacy |
| - fastapi | - pandas | - Scikit-Image | - tqdm |
| - requests | - numpy | - Tesseract | - typer |
| - jupyter | - matplotlib | - Imageio | - textblob |
| - tornado | - scipy | - PyWavelets | - wasabi |
| - click | - pillow | - imutils | - smart-open |
| - loguru | - seaborn | - networkx | - thinc |
| - pydantic | - cython | - packaging | - polyglot |
| - uvloop | + 4 packages2 | - [imageio] | + 14 packages3 |
| - jinja2 | - tifffile | ||
| + 77 packages4 |
The four images may be retrieved ´as-is´ or combined with a deep learning framework such as Tensorflow or PyTorch. Furthermore, the variants may be bundled with CUDA support.
The tables below give an overview of the 20 different variants of the images. Use the --image option followed by the image name shown in the tables below to specify the image you would like to use.
Images without CUDA
| No deep learning framework | Tensorflow | PyTorch | |
|---|---|---|---|
| Base | base | - | - |
| Slim | slim | tf-slim | torch-slim |
| Computer Vision | cv | tf-cv | torch-cv |
| Natural Language Processing | nlp | tf-nlp | torch-nlp |
CUDA-enabled images
| No deep learning framework | Tensorflow | PyTorch | |
|---|---|---|---|
| Base | cuda-base | - | - |
| Slim | cuda-slim | cuda-tf-slim | cuda-torch-slim |
| Computer Vision | cuda-cv | cuda-tf-cv | cuda-torch-cv |
| Natural Language Processing | cuda-nlp | cuda-tf-nlp | cuda-torch-nlp |
Custom images
You can use a non-Emily image by picking the custom image option when selecting an image.
This allows you to either use an image you have created yourself, or to use a publicly available one.
For example, to use the official Python image, you can use the flag --image python when running emily build.
Note that if you specify a non-existent or faulty image, the Emily project may fail to start once built.
Footnotes
-
Additional packages for the
cvimage: libsm6, libxext6, libglib2.0-0, libsm6, libxrender1, libfontconfig1, libxext6, libgl1 ↩ -
Additional python packages for the
slimimage: joblib, kiwisolver, threadpoolctl, cycler ↩ -
Additional python packages for the
nlpimage: blis, catalogue, corenlp, cymem, langcodes, lemmy, morfessor, murmurhash, pathy, preshed, pycld2, pyicu, regex, srsly ↩ -
Additional python packages for the
baseimage:psutil, gunicorn, prompt-toolkit, starlette, uvicorn, python-dotenv, pycodestyle, attrs, websockets, autopep8, pyzmq, urllib3, websocket-client, prometheus-client, pycparser, bleach, flake8, lxml, mistune, PyYAML, python-dateutil, pyrsistent, pyparsing, beautifulsoup4, grpcio, grpcio-tools, pyflakes, babel, pygments, anyio, watchfiles, toml, debugpy, six, decorator, certifi, traitlets, parso, mccabe, pytz, MarkupSafe, pandocfilters, packagingnest-asyncio, argon2-cffi, fastjsonschema, terminado, defusedxml, wcwidth, charset-normalizer, Send2Trash, cffi, executing, ptyprocess, soupsieve, asttokens, json5, tinycss2, typing-extensions, sniffio, entrypoints, pickleshare, webencodings, stack-data, pure-eval, backcall, argon2-cffi-bindings, pexpect, h11, httptools, idna, ipykernel, ipython-genutils, ipywidgets, jedi, jsonschema ↩