Running LIT in a Docker container#
Users might want to deploy LIT onto servers for public-facing, long-running instances. This is how we host the LIT demos found on https://pair-code.github.io/lit/demos/. This doc describes the basic usage of LIT’s built-in demos, how to integrate your custom demo into this
Basic Usage#
LIT can be run as a containerized app using Docker or your preferred engine. This is how we run our hosted demos.
We provide a basic
Dockerfile
that you can
use to build and run any of the demos in the lit_nlp/examples
directory. The
Dockerfile
installs all necessary dependencies for LIT and builds the
front-end code from source. Then it runs gunicorn as
the HTTP server, invoking the get_wsgi_app()
method from our demo file to get
the WSGI app to serve. The options provided to gunicorn for our use-case can be
found in
gunicorn_config.py
.
You can find a reference implementation in
glue_demo.py
or
lm_demo.py
.
Use the following shell commands to build the default Docker image for LIT from
the provided Dockerfile
, and then run a container from that image. Comments
are provided in-line to help explain what each step does.
# Build the docker image using the -t argument to name the image. Remember to
# include the trailing . so Docker knows where to look for the Dockerfile.
docker build --file Dockerfile --tag lit-nlp .
# Now you can run LIT as a containerized app using the following command. Note
# that the last parameter to the run command is the value you passed to the -t
# argument in the build command above.
docker run --rm -p 5432:5432 lit-nlp
The image above defaults to launching the GLUE demo on port 5432, but you can override this using the DEMO_NAME and DEMO_PORT environment variables, as shown below.
# DEMO_NAME is used to complete the Python module path
#
# "lit_nlp.examples.$DEMO_NAME"
#
# Therefore, valid values for DEMO_NAME are Python module paths in the
# lit_nlp/examples directory, such as
#
# * direct children -- glue_demo, lm_demo, image_demo, t5_demo, etc.
# * And nested children -- coref.coref_demo, is_eval.is_eval_demo, etc.
docker run --rm -p 5432:5432 -e DEMO_NAME=lm_demo lit-nlp
# Use the DEMO_PORT environment variable as to change the port that LIT uses in
# the container. Be sure to also change the -p option to map the container's
# DEMO_PORT to a port on the host system.
docker run --rm -p 2345:2345 -e DEMO_PORT=2345 lit-nlp
# Bringing this all together, you can run multiple LIT apps in separate
# containers on your machine using the combination of the DEMO_NAME and
# DEMO_PORT arguments, and docker run with the -d flag to run the container in
# the background.
docker run -d -p 5432:5432 -e DEMO_NAME=t5_demo lit-nlp
docker run -d -p 2345:2345 -e DEMO_NAME=lm_demo -e DEMO_PORT=2345 lit-nlp
Integrating Custom LIT Instances with the Default Docker Image#
Many LIT users create their own custom LIT server script to demo or serve, which
involves creating an executable Python module with a main()
method, as
described in the Python API docs.
These custom server scripts can be easily integrated with LIT’s default image as long as your server meets two requirements:
Ensure your server script is located in the
lit_nlp/examples
directory (or in a nested directory underlit_nlp/examples
).Ensure that your server script defines a
get_wsgi_app()
function similar to the minimal example shown below.
def get_wsgi_app() -> Optional[dev_server.LitServerType]:
"""Return WSGI app for container-hosted demos."""
# Set any flag defaults for this LIT instance
FLAGS.set_default("server_type", "external")
FLAGS.set_default("demo_mode", True)
# Parse flags before calling main()
unused = flags.FLAGS(sys.argv, known_only=True)
if unused:
logging.info("get_wsgi_app() called with unused args: %s", unused)
return main([])
Assuming your custom script meets the two requirements above, you can simply
rebuild the default Docker image and run a container using the steps above,
ensuring that you pass the -e DEMO_NAME=your_server_script_path_here
to the
run command.
A more detailed description of the get_wsgi_app()
code can be found below.
def get_wsgi_app() -> Optional[dev_server.LitServerType]:
"""Returns a WSGI app for gunicorn to consume in container-hosted demos."""
# Start by setting any default values for flags your LIT instance requires.
# Here we set:
#
# server_type to "external" (required), and
# demo_mode to "True" (optional)
#
# You can add additional defaults as required for your use case.
FLAGS.set_default("server_type", "external")
FLAGS.set_default("demo_mode", True)
# Parse any parameters from flags before calling main(). All flags should
# defined using one of absl's flags.DEFINE methods.
#
# Note the use of the known_only=True parameter here. This ensures that only
# those flags that have been define using one of absl's flags.DEFINE methods
# will be parsed from the command line arguments in sys.argv. All unused
# arguments will be returned as a Sequence[str].
unused = flags.FLAGS(sys.argv, known_only=True)
# Running a LIT instance in a container based on the default Dockerfile and
# image will always produce unused arguments, because sys.argv contains the
# command and parameters used to run the gunicorn sever. While not stricly
# required, we recommend logging these to the console, e.g., in case you need
# to verify the value of an environment variable.
if unused:
logging.info("get_wsgi_app() called with unused args: %s", unused)
# Always pass an empty list to main() inside of get_wsgi_app() functions, as
# absl apps are supposed to use absl.flags to define any and all flags
# required to run the app.
return main([])
Building Your Own Image#
Coming soon.