I am here because we needed a solution to a concurrency problem, access to hundreds of thousands of websites, parse and process all of them asynchronously. I tried many options always with a main idea: Futures.
So coding in that way I made something similar to the following code:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 | //.................... //game_1: Int value indicating first game //game_n: Int value indicating game n val resultMatch:List[Future[List[Match]]] = (game_1 until game_n).map { matchnum => Future{ new ParserFootbalHTML("https://www.anyurl.com/..." + matchnum).finalMatch }}.toList //.............. resultMatch foreach(future => future onComplete { Thread.sleep(1) doComplete }) //............... def doComplete: PartialFunction[Try[List[Match]],Unit] = { // case matches @ Success(_) =>{ val resultMatch:List[Match] = matches.getOrElse(List.empty[Match]) resultMatch.foreach(matches=> //Do anything with matches ) } case fail @ Failure(_) => println("error") } //.................... |
With minimal changes the main idea was to launch as many "Futures" as web sites we need to parse. Then after every computation finish(onComplete) I will do "anything" with the result. So after several test in different escenarios checking how long time did it take I decided to explore Akka, for improve performance and reduce time of calculation.
The Core of Akka libraries is the Actor Model, I recommend understand it because is the base of this libraries. Akka has made its own implementation of this model.
- Actor and Actor-1 communicate exclusively by exchanging messages asynchronously which are placed into the recipient’s MailBox.
- Actor and Actor 1 should avoid hogging resources.
- Message SHOULD NOT be mutable.
ONE important thing from the official documentation:
The only meaningful way for a sender to know whether an interaction was successful is by receiving a business-level acknowledgement message, which is not something Akka could make up on its own (neither are we writing a “do what I mean” framework nor would you want us to).
So as we can see this is the real concept of encapsulation. Every thing happens inside the actor and if you want to delegate a task then It send a message to the actor and keep working. For that reason you ought granulate every task in actor as simple as possible and you should have at the end that each actor processing be as simple as possible.
Split up the task and delegate it until they become small enough to be handle in One piece!
fig 1.3
As show in fig 1.3 when you invoke the ActorSystem there are several actors that have already been created.
Actor1 is a Supervisor of Actor1.1 and Actor 1.2.
Create only ONE ActorSystem per application is considered a BestPractice because it is a very heavy object and can be create by different ways:
The ActorSystem created can be in the same context or in any other execution context created by us and declared implicitly.
As we can see in fig 1.3 there are different ways of creating actors:
- Actors that are children of user "guardian" are created by: system.actorOf and in this case I use to create not many of this kind of actors due to those actors will be the own root actor of my app.
- Actor that will be child of my own actors(created by us) are created by: context.actorOf , the context is an ActorContext implicit val that there is in the Actor trait.
- self()
- sender()
- watch(): used for track actor activities and know when it stopped. Expected a "Terminated"
- unwatch()
- actorOf()
- become(): change the actor behavior. We decide what message we processed and how. An actor can start processing some type of message and the change its behavior for any reason.
- actorSelection(actor path): We can select actor for its path (ref. fig 1.3 user/actor1, user/Actor1/Actor1.2)
Terminate(ActorRef that is being watched, ref watch). When it happen the Actor free up its resources.
When we have an ActorRef:
As best practice we should try to send message via ! or tell. [Explain it]
We indicate below the best way to create and invoke different types of actors:
............. object BootNotScheduled extends App with ScalaLogger{ val moviePageBoundaries: List[Int] = Try(List(args(0).toInt, args(1).toInt)) match { case Success(expectedList) => expectedList case Failure(e) => log.error("Error with the boundaries of your page numbers,reviews your parameters {}",e.toString) System.exit(1) Nil } // This actor will control the whole System ("IoTDaemon_Not_Scheduled") val system = ActorSystem(ConstantsObject.NotScheduledIoTDaemon) // creating ioTAdmin actor val ioTAdmin = system.actorOf(IoTAdmin.props, ConstantsObject.NotScheduledIoTDaemonProcessing) val filteredAndOrderedList = moviePageBoundaries.toSet.toList.sorted val filteredAndOrderedListGamesBoundaries = (filteredAndOrderedList.head to filteredAndOrderedList.last).toList // Sending a message to ioTAdmin actor ioTAdmin!ParseUrlFilms(filteredAndOrderedListGamesBoundaries,Option(ioTAdmin)) }
Props are config classes to specify options when you create actors. You can find below an snapshot configuring the actor ioTAdmin previously created in the above code:
........ object IoTAdmin { case class ParseUrlFilms(listTitles: List[moviePage], actorRef: Option[ActorRef]=None) case object ErrorStopActor case object StopProcess def props: Props = Props(new IoTAdmin) } ... override def receive = { ..... }
The way how I deal with the configuration is similar as used to be done in most libraries or framework or even in java language. I read the config info from application.conf but there are several way for do that you can find it in Akka config information, it does not explain at all how deal with it.
What do you have to consider when deal with configuration files in akka:
- Akka configuration values. We will use default values for the moment.
- Where the configuration file should be saved and how name it.
- How the configuration files are loaded or read. [related class/interfaces: ConfigFactory, Config]
The configuration files follow the HOCON specification and we can find a very simple examples on github that will let us create config files easily.
The main akka config files use to be application.conf and uses to be placed in the "resource" folder.
We already have minimal information to create an Akka App only using the actor module anyway you can have a look to my github repository akka-quickstart-scala and check about how install and launching it(try the first option in your terminal[sbt "AkkaSchedulerPoc/runMain com.ldg.BootNotScheduled 1 5")]
Router
Dispatcher
Akka Test Kit
Akka HTTP
Entradas
