Running LIT in a Docker container¶

Users might want to deploy LIT onto servers for public-facing, long-running instances. This is how we host the LIT demos found on https://pair-code.github.io/lit/demos/. This doc describes the basic usage of LIT’s built-in demos, how to integrate your custom demo into this

Basic Usage¶

LIT can be run as a containerized app using Docker or your preferred engine. This is how we run our hosted demos.

We provide a basic Dockerfile https://github.com/PAIR-code/lit/blob/main/Dockerfile that you can use to build and run any of the demos in the lit_nlp/examples directory. The Dockerfile installs all necessary dependencies for LIT and builds the front-end code from source. Then it runs gunicorn as the HTTP server, invoking the get_wsgi_app() method from our demo file to get the WSGI app to serve. The options provided to gunicorn for our use-case can be found in gunicorn_config.py. You can find a reference implementation in glue/demo.py.

Use the following shell https://github.com/PAIR-code/lit/blob/main/.github/workflows/ci.yml commands to build the default Docker image for LIT from the provided Dockerfile, and then run a container from that image. Comments are provided in-line to help explain what each step does.

# Build the docker image using the -t argument to name the image. Remember to
# include the trailing . so Docker knows where to look for the Dockerfile.
docker build --file Dockerfile --tag lit-nlp .

# Now you can run LIT as a containerized app using the following command. Note
# that the last parameter to the run command is the value you passed to the -t
# argument in the build command above.
docker run --rm -p 5432:5432 lit-nlp

The image above defaults to launching the GLUE demo on port 5432, but you can override this using the DEMO_NAME and DEMO_PORT environment variables, as shown below.

# DEMO_NAME is used to complete the Python module path
#
#     "lit_nlp.examples.$DEMO_NAME.demo:get_wsgi_app()"
#
# Therefore, valid values for DEMO_NAME are Python module paths in the
# lit_nlp/examples directory, such as glue, penguin, tydi, etc.
docker run --rm -p 5432:5432 -e DEMO_NAME=penguin lit-nlp

# Use the DEMO_PORT environment variable as to change the port that LIT uses in
# the container. Be sure to also change the -p option to map the container's
# DEMO_PORT to a port on the host system.
docker run --rm -p 2345:2345 -e DEMO_PORT=2345 lit-nlp

# Bringing this all together, you can run multiple LIT apps in separate
# containers on your machine using the combination of the DEMO_NAME and
# DEMO_PORT arguments, and docker run with the -d flag to run the container in
# the background.
docker run -d -p 5432:5432 -e DEMO_NAME=penguin lit-nlp
docker run -d -p 2345:2345 -e DEMO_NAME=tydi -e DEMO_PORT=2345 lit-nlp

Integrating Custom LIT Instances with the Default Docker Image¶

Many LIT users create their own custom LIT server script to demo or serve, which involves creating an executable Python module with a main() method, as described in the Python API docs.

These custom server scripts can be easily integrated with LIT’s default image as long as your server meets two requirements:

  1. Ensure your server script is located in the lit_nlp/examples directory (or in a nested directory under lit_nlp/examples).

  2. Ensure that your server script defines a get_wsgi_app() function similar to the minimal example shown below.

def get_wsgi_app() -> Optional[dev_server.LitServerType]:
  """Return WSGI app for container-hosted demos."""
  # Set any flag defaults for this LIT instance
  FLAGS.set_default("server_type", "external")
  FLAGS.set_default("demo_mode", True)
  # Parse flags before calling main()
  unused = flags.FLAGS(sys.argv, known_only=True)
  if unused:
    logging.info("get_wsgi_app() called with unused args: %s", unused)
  return main([])

Assuming your custom script meets the two requirements above, you can simply rebuild the default Docker image and run a container using the steps above, ensuring that you pass the -e DEMO_NAME=your_server_script_path_here to the run command.

A more detailed description of the get_wsgi_app() code can be found below.

def get_wsgi_app() -> Optional[dev_server.LitServerType]:
  """Returns a WSGI app for gunicorn to consume in container-hosted demos."""
  # Start by setting any default values for flags your LIT instance requires.
  # Here we set:
  #
  #     server_type to "external" (required), and
  #     demo_mode to "True" (optional)
  #
  # You can add additional defaults as required for your use case.
  FLAGS.set_default("server_type", "external")
  FLAGS.set_default("demo_mode", True)

  # Parse any parameters from flags before calling main(). All flags should
  # defined using one of absl's flags.DEFINE methods.
  #
  # Note the use of the known_only=True parameter here. This ensures that only
  # those flags that have been define using one of absl's flags.DEFINE methods
  # will be parsed from the command line arguments in sys.argv. All unused
  # arguments will be returned as a Sequence[str].
  unused = flags.FLAGS(sys.argv, known_only=True)

  # Running a LIT instance in a container based on the default Dockerfile and
  # image will always produce unused arguments, because sys.argv contains the
  # command and parameters used to run the gunicorn sever. While not stricly
  # required, we recommend logging these to the console, e.g., in case you need
  # to verify the value of an environment variable.
  if unused:
    logging.info("get_wsgi_app() called with unused args: %s", unused)

  # Always pass an empty list to main() inside of get_wsgi_app() functions, as
  # absl apps are supposed to use absl.flags to define any and all flags
  # required to run the app.
  return main([])

Building Your Own Image¶

Coming soon.