Generating scala application distribution in akka & play for different environments

I have to much for talk about sbt. I have been using this build tooling for a long time and honestly every time that I have to do something  different I need to check sbt documentation. I’ve been thinking in something like sbt in a nutshell, ha ha. Forget about it!. At the same time too many people think that SBT is confusing, complicated, and opaque  and there are a lot of explanation about why. I won't do that in this article.
My target today: how generate distribution package for different environments and the most important  just in one command line clean + compile + build distribution +specific environment

What we need:

1. pass parameters to our process indicating what environment do you want to build. 
2. create a plugin to read the parameters. 
3. create a task that will let you add to the project the appropriate configuration files before the compilation process. 
4. create a command that have to package the distribution with the suitable option.

I am going to use sbt native packager  for generate distribution for different environments. You don't need to be familiar with this libraries for understand this post but if you want to generate other distributions different than we have explained here my recommendation is a learning in-depth of it.

How to solve point 1:

The first thing that we need to do is to pass a variable in our building process:

sbt -Dour_system_property_var=value_to_assign

So in our case we need to indicate the environment that we are building for:
I am gonna read this environment and set the proper configuration file that will be added to our jar distributions.

You need to coding what to do, considering the parameters that has been passed. You can take some reference about SBT parameters and Build Environment.

Our next steps :

1. read variables to indicate what environment we want the generation of the new package distribution.
2. depending of our parameter we need to indicate what config files we want to use in our compilation process. 

To carry out the aforementioned process we need to create an sbt plugin(BuildEnvPlugin.scala):

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49

import sbt.Keys._ import sbt._ import sbt.plugins.JvmPlugin /** * make call in this way for generate development distro * sbt -Denv=dev reload clean compile stage * * make call in this way for generate production distro * sbt -Denv=prod reload clean compile stage * */ /** sets the build environment */ object BuildEnvPlugin extends AutoPlugin { // make sure it triggers automatically override def trigger = AllRequirements override def requires = JvmPlugin object autoImport { object BuildEnv extends Enumeration { val Production, Test, Development = Value } val buildEnv = settingKey[BuildEnv.Value]("the current build environment") } import autoImport._ override def projectSettings: Seq[Setting[_]] = Seq( buildEnv := { sys.props.get("env") .orElse(sys.env.get("BUILD_ENV")) .flatMap { case "prod" => Some(BuildEnv.Production) case "test" => Some(BuildEnv.Test) case "dev" => Some(BuildEnv.Development) case unkown => None } .getOrElse(BuildEnv.Development) }, // message indicating in what environment you are building on onLoadMessage := { val defaultMessage = onLoadMessage.value val env = buildEnv.value s"""|$defaultMessage |Running in build environment: $env""".stripMargin } ) }

code 1.0

The plugin code above return some build environment that need to be read for any other process.  In a first solution we are going to solve this problem in the main build.sbt file.

In our main build.sbt file (snapshot of code below) depending of the environment selected we are going to copy the proper configuration file to conf/application.conf configuration file  @see line 9 in code below, code 1.1, pay attention that buildEnv.value in line 4 is created/loaded in our customized sbt plugin (code above, code 1.0)

1 2 3 4 5 6 7 8 9 10 11 12

............. mappings in Universal += { val confFile = buildEnv.value match { case BuildEnv.Development => "development.conf" case BuildEnv.Test => "test.conf" case BuildEnv.Production => "production.conf" } ((resourceDirectory in Compile).value / confFile) -> "conf/application.conf" } .............

code 1.1

Once that you have selected the environment we need to copy all suitable configuration files, afterwards we need to add this code to a task.
Just in this point can pass parameters to our process, indicating what environment we want to build, we have code a plugin than can read parameters. We need to create a command as simple as possible CommandScalaPlay.scala, it will let us to execute in one line reload + clean + compile + tgz distribution of our project.

1 2 4 3 4 5 6 7 8 9 10 11 12 13 14 15

import sbt._ object CommandScalaPlay { /** https://www.scala-sbt.org/sbt-native-packager/gettingstarted.html#native-tools */ // A simple, multiple-argument command that prints "Hi" followed by the arguments. // Again, it leaves the current state unchanged. // launching from console a production building: sbt -Denv=prod buildAll def buildAll = Command.args("buildAll", "<name>") { (state, args) => println("buildAll command generating tgz in Universal" + args.mkString(" ")) "reload"::"clean"::"compile"::"universal:packageZipTarball":: state } }

code 1.2

In the above code in code 1.2 we have create a very simple command that   reload + clean + compile + tgz distribution of our project in line 11. You can glance over sbt-native-packager output format in universal if  you are interested in other output format different than tgz.

We need to add the command create above in code 1.3 in our build.sbt main file. So in code below we add the command @see line 11 in code below, code 1.3.

1 2 3 4 5 6 7 8 9 10 11 12

lazy val commonSettings = Seq( name:= """sport-stats""", organization := "com.mojitoverdeintw", version := "1.0", scalaVersion := "2.11.7" ) ................ settings(commonSettings, resourceDirectory in Compile := baseDirectory.value / "conf", commands ++= Seq(buildAll)) ................

code 1.3

In code above we say to the compiler that our resource directory will be /conf @see line 10 in code above, code 1.3. It shouldn't be necessary because resource directory( resourceDirectory in sbt) in Scala apps sbt resourceDirectory point to src/main/resources but in play framework should be point to /conf.

If you want to create an specific distribution:

go to your HOME_PROJECT and then:

1. a tgz for production env sbt -Denv=prod buildAll
2. a tgz for development env sbt -Denv=dev buildAll
3. a universal for development env sbt -Denv=dev stage
4. a universal for production env sbt -Denv=dev stage

If you want to implement the above scenario in your own project you need take a look at the following points:

1. sbt version: in the above scenario is 0.13.11,  reference file: build.properties in my github repository.
2. plugins.sbt(in my github repository): pay attention to this line: addSbtPlugin("com.typesafe.sbt" % "sbt-native-packager" % "1.3.2") in my github repository.
3. build.sbt(in my github repository): Pay attention to this file, mainly about code 1.1 y code 1.4 below in this post.

Take a look about image below where should be the aforementioned files (CommandScalaPlay.scala, BuildEnvPlugin.scala, plugins.sbt)

                                         fig 1.1

Remember that in our configuration folder(in playframework) /conf should be all configuration files that we want in different environments during the distro generation process so at the end one of those files will be our application.conf configuration file. 

In other scenario Instead of create the application.conf file in our main build.sbt file, I reckon that it should be done in code because build.sbt should be as clean as possible. 

                                          fig 1.2

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

import sbt._ import sbt.Keys._ import sbt.plugins.JvmPlugin /** * make call in this way for generate development distro * sbt -Denv=dev reload clean compile stage * * make call in this way for generate production distro * sbt -Denv=prod reload clean compile stage * */ /** sets the build environment */ object BuildEnvPlugin extends AutoPlugin { ............................ lazy val configfilegeneratorTask = Def.task { val confFile = buildEnv.value match { case BuildEnv.Development => "development.conf" case BuildEnv.Test => "test.conf" case BuildEnv.Stage => "stage.conf" case BuildEnv.Production => "production.conf" } val filesrc = (resourceDirectory in Compile).value / confFile val file = (resourceManaged in Compile).value / "application.conf" IO.copyFile(filesrc,file) Seq(file) } ..................... }

code 1.4

Pay special attention to the task definition configfilegeneratorTask in line 19  in above code(code 1.4) in BuildEnvPlugin.scala(in my github repository)  and then modify the build.sbt file. See the code below and changes that have to be done in our main build.sbt(in my github repository).

1 2 3 4 5 6 7 8 9 10 11 12 13

................... lazy val root = (project in file(".")). aggregate(internetOfThings). aggregate(AkkaSchedulerPoc). settings(commonSettings,commands ++= Seq(buildAll)) lazy val AkkaSchedulerPoc = (project in file("modules/AkkaScheduler")). settings(commonSettings,libraryDependencies ++= Seq( "com.enragedginger" %% "akka-quartz-scheduler" % "1.6.1-akka-2.5.x" ),resourceGenerators in Compile += configfilegenerator) ...................

code 1.5

Have a look in above code(code 1.5) line 11, here we're adding our task configfilegenerator in our sbt compilation process. You can have a look to this files: build.properties, plugins.sbt, build.sbt

The reference about the last scenario is hosted in other project different than the first scenario but

Remember that plugin and task should be under root project folder. (fig 1.1 above in our post)

Remember that in our configuration folder (in akka) /resource folder  /conf should be all configuration files that we want in different environments during the distro creation process so at the end one of those files will be our application.conf configuration file. 

                                                 fig 1.3
                                 

You need to keep in mind when you are working with autoplugin, when you create task o the other object that execute any processes in an specific phase that you could face some problems about initialization process because some setting are loaded twice. So the sbt initialization could be a useful reading.
In next posts I will explain through code other important implementation when we are working with sbt.

There are different ways to generate distributions in different environments, SBT has many other solutions, but in my opinion they are specific to each file and not for a complete configuration like (log specification file + application configuration file + other properties or metadata file that can change when we change environment)

In this Post we show you examples in Playframework and Akka but how to solve if you want to generate any implementation or distribution without any plugins so you can choose the options that SBT give you:

• sbt -Dconfig.file=<Path-to-my-specific-config-file>/my-environment-conf-file.conf other-phases 

An example about the aforementioned statement:

     sbt -Dconfig.file=/home/myusr/development.conf reload clean compile stage 

In the same way you can use -Dconfig.resource so in this case you don’t need to specify the path to the file. Sbt will try to find out in the resource directory.

You can do the same with logging configuration so in the same way:

• sbt -Dlogger.file  <Path-to-my-specific-logger-config-file>/my-environment-logger-conf-file.conf  

or like in config.resource you can use -Dlogger.resource  and in this case you don’t need to specify the path to the log configuration file 'cos the file will be loaded from the classpath.

There are some interesting links in Sbt explaining how deal with distro generation process in different environments :

1. Specifying different configurations files. 
2. Specifying different loggings configurations files. 

Something to keep in mind: to use configurations files properly pay attention to HOCON (Human-Optimized Config Object Notation) and how it work. It could be very helpful. We can tackle this subject in next posts

We agreed that SBT is not easy, it is a powerful tool but it is not easy. Perhaps for that reason developer don't use it if they can use any other build tooling. Any way I will be posting some problems that we face when we are coding with scala and SBT make easier our work.