R Model Deployment by Corey-Zumar · Pull Request #261 · ucbrise/clipper

Corey-Zumar · 2017-07-26T08:07:05Z

Note: This depends on #251. Rebasing after it is merged will significantly reduce the file diff. This is functional, but it still needs documentation.

AmplabJenkins · 2017-07-26T08:10:13Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/593/
Test FAILed.

AmplabJenkins · 2017-07-26T08:15:12Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/594/
Test FAILed.

dcrankshaw

This looks like it was a pain to write, but the end result is great!

dcrankshaw · 2017-07-28T00:34:30Z

containers/R/rclipper_serve/R/r_clipper.R

+        return(deserialized_pred_fn)
+      }
+      input_class = class(deserialized_input)
+      if(input_class != input_class) {


This looks like a hard comparison to fail...

Fixed. We now correctly compare the class of the deserialized input received by the container to the class of inputs expected to be received by the prediction function. These should match.

dcrankshaw · 2017-07-28T01:06:13Z

During code review, we came up with a few different design choices for how the client-side model deployment package should work.

Current implementation: Serialize the user's function and then drop into Python code which calls clipper_admin.ClipperConnection.build_and_deploy_model(). However, there is no existing container manager instance, so the user needs to provide enough information for the underlying Python script to create a new container manager instance. This includes the type of container manager (docker vs k8s for now), and the corresponding arguments for ContainerManager.connect() (e.g. k8s cluster address, query and management frontend ports for Docker, etc). This is fairly complete, but leads to a pretty heavyweight interface.
Make the interface as minimal as possible, by just providing enough information to build the model container and push to a registry. This could be implemented in pure R with a few system calls to Docker commands (docker build, docker push). However, in order to actually deploy this model, the user would need to separately call clipper_admin.ClipperConnection.deploy_model from Python, providing a ContainerManager instance. One thing that might trip up users with this approach is correctly specifying the input_type argument, but we could print out exactly what command to invoke when calling deploy_model.

AmplabJenkins · 2017-09-01T09:07:27Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/675/
Test FAILed.

AmplabJenkins · 2017-09-01T09:12:37Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/676/
Test FAILed.

AmplabJenkins · 2017-09-01T09:17:41Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/677/
Test FAILed.

AmplabJenkins · 2017-09-01T09:45:05Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/678/
Test FAILed.

AmplabJenkins · 2017-09-01T10:26:53Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/679/
Test PASSed.

dcrankshaw

Cool. Looks good. Add tests and update the Docker file and I'll try it out.

dcrankshaw · 2017-09-03T20:42:21Z

containers/R/RContainerDockerfile

@@ -0,0 +1,24 @@
+FROM r-base


Add the following two lines above the FROM

# This ARG isn't used but prevents warnings in the build script ARG CODE_VERSION

dcrankshaw · 2017-09-03T20:42:48Z

containers/R/RContainerDockerfile

+RUN R CMD INSTALL rclipper_serve
+
+ENTRYPOINT ["/entrypoint.sh"]
+


Move this to the dockerfiles/ subdirectory and add a call to build it to bin/build_docker_images.sh

dcrankshaw · 2017-09-03T20:43:39Z

containers/R/RContainerDockerfile

+
+COPY rclipper_serve rclipper_serve
+COPY serve_model.R serve_model.R
+COPY r_container_entrypoint.sh entrypoint.sh


Don't change the name of the file when you copy it (just to make debugging a little easier)

dcrankshaw · 2017-09-03T20:44:21Z

containers/R/RContainerDockerfile

+ENV CLIPPER_MODEL_PATH=/model
+ENV CLIPPER_PORT=7000
+
+RUN R -e "install.packages('jsonlite', repos='http://cran.us.r-project.org')" 


Combine these into a single RUN command

dcrankshaw · 2017-09-03T20:45:03Z

containers/R/rclipper_serve/DESCRIPTION

+Version: 1.0
+Date: 2017-07-15
+Author: Corey Zumar
+Maintainer: Dan Crankshaw <crankshaw@eecs.berkeley.edu>


Make yourself the maintainer

dcrankshaw · 2017-09-03T22:24:05Z

containers/R/rclipper_user/DESCRIPTION

+Version: 1.0
+Date: 2017-07-22
+Author: Corey Zumar
+Maintainer: Dan Crankshaw <crankshaw@eecs.berkeley.edu>


Make yourself maintainer

dcrankshaw · 2017-09-03T22:24:13Z

containers/R/rclipper_user/DESCRIPTION

+Package: rclipper
+Type: Package
+Title: Containerizes and deploys R closures to Clipper
+Version: 1.0


Version 0.1

dcrankshaw · 2017-09-03T22:24:54Z

containers/R/rclipper_user/R/dependencies.R

@@ -0,0 +1,206 @@
+get_library_dependencies = function(cd_dependencies) {


This should definitely have some tests

dcrankshaw · 2017-09-03T22:26:38Z

containers/R/rclipper_user/inst/launch_container.py

@@ -0,0 +1,76 @@
+import sys


Update this file to match the latest version of clipper_admin.

dcrankshaw · 2017-09-03T22:28:05Z

containers/R/rclipper_user/inst/launch_container.py

+
+    print("Building image and deploying model...")
+
+    cl_conn.deploy_model(args.model_name, args.model_version, input_type,


This should be build_model. As discussed, let's just build the model in this script and not deploy it.

AmplabJenkins · 2017-09-07T02:40:28Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/703/
Test FAILed.

Corey-Zumar

Addressed most of the review comments. This still needs unit tests and documentation. I also still need to include the R container integration test in our integration test suite.

Corey-Zumar · 2017-09-07T04:14:16Z

containers/R/RContainerDockerfile

@@ -0,0 +1,24 @@
+FROM r-base


Corey-Zumar · 2017-09-07T04:16:08Z

containers/R/rclipper_user/DESCRIPTION

+Package: rclipper
+Type: Package
+Title: Containerizes and deploys R closures to Clipper
+Version: 1.0


Corey-Zumar · 2017-09-07T04:16:10Z

containers/R/rclipper_user/DESCRIPTION

+Version: 1.0
+Date: 2017-07-22
+Author: Corey Zumar
+Maintainer: Dan Crankshaw <crankshaw@eecs.berkeley.edu>


Corey-Zumar · 2017-09-07T04:16:26Z

containers/R/RContainerDockerfile

+
+COPY rclipper_serve rclipper_serve
+COPY serve_model.R serve_model.R
+COPY r_container_entrypoint.sh entrypoint.sh


Corey-Zumar · 2017-09-07T04:16:30Z

containers/R/RContainerDockerfile

+ENV CLIPPER_MODEL_PATH=/model
+ENV CLIPPER_PORT=7000
+
+RUN R -e "install.packages('jsonlite', repos='http://cran.us.r-project.org')" 


Corey-Zumar · 2017-09-07T04:26:21Z

containers/R/RContainerDockerfile

+RUN R CMD INSTALL rclipper_serve
+
+ENTRYPOINT ["/entrypoint.sh"]
+


Corey-Zumar · 2017-09-07T04:30:23Z

containers/R/rclipper_serve/R/r_clipper.R

+        return(deserialized_pred_fn)
+      }
+      input_class = class(deserialized_input)
+      if(input_class != input_class) {


Fixed. We now correctly compare the class of the deserialized input received by the container to the class of inputs expected to be received by the prediction function. These should match.

Corey-Zumar · 2017-09-07T04:30:26Z

containers/R/rclipper_serve/R/r_clipper.R

+      if(input_class != input_class) {
+        return(sprintf("Received invalid input of class `%s`` for model expecting inputs of class `%s`", 
+                       input_class, 
+                       input_class))


AmplabJenkins · 2017-09-07T04:43:56Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/704/
Test FAILed.

AmplabJenkins · 2017-09-07T05:26:26Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/705/
Test PASSed.

AmplabJenkins · 2017-09-08T01:13:18Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/706/
Test FAILed.

AmplabJenkins · 2017-09-08T01:46:03Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/707/
Test FAILed.

AmplabJenkins · 2017-09-08T08:23:04Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/708/
Test FAILed.

AmplabJenkins · 2017-09-08T08:27:35Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/709/
Test FAILed.

AmplabJenkins · 2017-09-08T21:44:19Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/711/
Test FAILed.

AmplabJenkins · 2017-09-08T22:48:28Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/713/
Test FAILed.

AmplabJenkins · 2017-09-08T22:53:10Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/714/
Test FAILed.

AmplabJenkins · 2017-09-08T22:57:40Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/715/
Test FAILed.

AmplabJenkins · 2017-09-08T23:02:11Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/716/
Test FAILed.

AmplabJenkins · 2017-09-08T23:06:42Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/717/
Test FAILed.

AmplabJenkins · 2017-09-12T20:14:26Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/736/
Test FAILed.

AmplabJenkins · 2017-09-12T20:32:13Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/737/
Test FAILed.

shaneknapp · 2017-09-12T20:33:25Z

i killed the job as the worker was inundated w/zombie procs. i'll kick off a new one now.

shaneknapp · 2017-09-12T20:33:30Z

test this please

dcrankshaw · 2017-09-12T20:37:57Z

jenkins test this please

AmplabJenkins · 2017-09-12T21:19:24Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/738/
Test FAILed.

AmplabJenkins · 2017-09-12T22:04:37Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/739/
Test FAILed.

AmplabJenkins · 2017-09-12T23:24:22Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/740/
Test FAILed.

AmplabJenkins · 2017-09-13T00:32:46Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/741/
Test FAILed.

AmplabJenkins · 2017-09-14T06:53:59Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/746/
Test FAILed.

AmplabJenkins · 2017-09-14T07:36:33Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/747/
Test FAILed.

dcrankshaw · 2017-09-14T14:36:38Z

Jenkins test this please

AmplabJenkins · 2017-09-14T15:22:38Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/748/
Test FAILed.

AmplabJenkins · 2017-09-14T17:24:37Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/749/
Test PASSed.

dcrankshaw

Fixed a final bug where Rcpp::NumericVector is equivalent to std::vector<double> not std::vector<float>. Once the tests pass with this change, this is good to go.

AmplabJenkins · 2017-09-14T23:04:41Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/750/
Test PASSed.

Corey-Zumar · 2017-09-15T00:06:46Z

Awesome! Thanks Dan!

Corey-Zumar added the status: in progress label Jul 26, 2017

dcrankshaw requested changes Jul 28, 2017

View reviewed changes

Corey-Zumar force-pushed the r_deploy branch from 969b866 to ff8b879 Compare September 1, 2017 08:46

dcrankshaw requested changes Sep 3, 2017

View reviewed changes

dcrankshaw added status: needs revision type: enhancement and removed status: in progress labels Sep 3, 2017

Corey-Zumar force-pushed the r_deploy branch from 5478518 to e2089ba Compare September 7, 2017 02:30

Corey-Zumar commented Sep 7, 2017

View reviewed changes

fix imports

c5795f2

install clipper_admin first

bcc8acd

fix path

7d1d2c2

fix path

31e7031

dcrankshaw added 2 commits September 14, 2017 06:06

Fixed unit tests

a634ae3

format code

2031dbf

Merge branch 'develop' into r_deploy

87a375a

Fixed typo

99b7e9b

Fixed mapping from R numeric to clipper doubles

9ea0c2b

dcrankshaw approved these changes Sep 14, 2017

View reviewed changes

dcrankshaw added status: accepted and removed status: needs review labels Sep 14, 2017

dcrankshaw merged commit b4ed00f into ucbrise:develop Sep 14, 2017

dcrankshaw mentioned this pull request Oct 3, 2017

First class support for deploying R models #242

Closed

		RUN R CMD INSTALL rclipper_serve

		ENTRYPOINT ["/entrypoint.sh"]

		@@ -0,0 +1,206 @@
		get_library_dependencies = function(cd_dependencies) {


		print("Building image and deploying model...")

		cl_conn.deploy_model(args.model_name, args.model_version, input_type,

Conversation

Corey-Zumar commented Jul 26, 2017

Uh oh!

AmplabJenkins commented Jul 26, 2017

Uh oh!

AmplabJenkins commented Jul 26, 2017

Uh oh!

dcrankshaw left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dcrankshaw commented Jul 28, 2017

Uh oh!

AmplabJenkins commented Sep 1, 2017

Uh oh!

AmplabJenkins commented Sep 1, 2017

Uh oh!

AmplabJenkins commented Sep 1, 2017

Uh oh!

AmplabJenkins commented Sep 1, 2017

Uh oh!

AmplabJenkins commented Sep 1, 2017

Uh oh!

dcrankshaw left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AmplabJenkins commented Sep 7, 2017

Uh oh!

Corey-Zumar left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!