ScalaLoci

A programming language for distributed applications

Features

Unified
Implement all components of a distributed application in a single language
Universal
Freely express any distributed architecture
Safe
Enjoy static type-safety across components and static checks for architectural constraints

Concepts

Specify Architecture

Define the architectural relation of the components of the distributed system

@peer type Server <: {
  type Tie <: Multiple[Client]
}

@peer type Client <: {
  type Tie <: Single[Server]
}

Specify Placement

Control where data is located and computations are executed

val items: Items on Server = placed {
  getCurrentItems()
}

val ui: UI on Client = placed {
  new UI
}

Compose

Combine data flow across components through reactive programming

val items: Var[Items] on Server =
  placed { Var(getCurrentItems()) }

on[Client] {
  Signal {
    items.asLocal() map createUIEntry
  }
}

Step-by-Step Example

The example demonstrates the design of a chat application in five steps

  1. Declaring a Server
  2. Declaring a Client
  3. Declaring a message event on the Client and firing the event for every user input
  4. Declaring a publicMessage event on the Server aggregating all events from the clients
  5. Declaring a client-side observer for the server-side publicMessage event
@multitier object Chat {
  @peer type Server <: { type Tie <: Multiple[Client] }













}
Server declaration


  @peer type Client <: { type Tie <: Single[Server] }













Client declaration




  val message = on[Client] { Evt[String]() }





  def main() = on[Client] {

    for (line <- io.Source.stdin.getLines)
      message.fire(line)
  }

Event message declaration






  val publicMessage = on[Server] {
    message.asLocalFromAllSeq map { case (_, message) => message }
  }







Event publicMessage declaration











    publicMessage.asLocal observe println




Remote observer for publicMessage declaration
Incremental Chat example

Getting Started

Try out ScalaLoci Examples from GitHub

Add ScalaLoci to Your Project

  1. Enable the Macro Paradise Plugin (for macro annotations) in your build.sbt
    addCompilerPlugin("org.scalamacros" % "paradise" % "2.1.1" cross CrossVersion.patch)
  2. Add the resolver for the ScalaLoci dependencies to your build.sbt
    resolvers += Resolver.bintrayRepo("stg-tud", "maven")
  3. Add the ScalaLoci dependencies that you need for your system to your build.sbt
    1. ScalaLoci language (always required):
      libraryDependencies += "de.tuda.stg" %% "scala-loci-lang" % "0.3.0"
    2. Transmitter for the types of values to be accessed remotely (built-in Scala types and standard collections are directly supported without additional dependencies)
      • REScala reactive events and signals
        libraryDependencies += "de.tuda.stg" %% "scala-loci-lang-transmitter-rescala" % "0.3.0"
    3. Network communicators to connect the different components of the distributed system
      • TCP [JVM only]
        libraryDependencies += "de.tuda.stg" %% "scala-loci-communicator-tcp" % "0.3.0"
      • WebSocket (using Akka HTTP on the JVM) [server: JVM only, client: JVM and JS web browser APIs]
        libraryDependencies += "de.tuda.stg" %% "scala-loci-communicator-ws-akka" % "0.3.0"
      • WebSocket (Play integration) [server: JVM only, client: JVM and JS web browser APIs]
        libraryDependencies += "de.tuda.stg" %% "scala-loci-communicator-ws-akka-play" % "0.3.0"
      • WebRTC [JS web browser APIs only]
        libraryDependencies += "de.tuda.stg" %% "scala-loci-communicator-webrtc" % "0.3.0"
    4. Serializer for network communication
      • µPickle serialization
        libraryDependencies += "de.tuda.stg" %% "scala-loci-serializer-upickle" % "0.3.0"
      • Circe serialization
        libraryDependencies += "de.tuda.stg" %% "scala-loci-serializer-circe" % "0.3.0"

Showcases

P2P Chat

The example logs messages that are sent and received by each participant as a composition of the data flow from the local UI and from remote chat partners. In the application, nodes are connected to multiple remote nodes and maintain a one-to-one chat with each. Users can select any chat to send messages. The messageSent event is defined as subjective value filtering the ui.​messageTyped messages from the UI for the currently active chat partner node. The messageLog signal contains the chat log for the chat between the local peer instance and the remote node given as parameter. It merges the remote stream for the chat messages from the remote instance node and the local stream subjective to the remote instance node via the || operator. The chat log is a signal created using list, which extends the list by an element for each new event occurrence in the merged stream. The chatLogs signal folds the remote[Node].​joined event stream, which is fired for each newly connected chat partner, into a signal that contains the chat logs for every chat partner generated by calling messageLog.

@multitier object Chat {
  @peer type Node <: { type Tie <: Multiple[Node] }

  val ui = on[Node] { UI() }

  val messageSent = on[Node] sbj { node: Remote[Node] =>
    ui.messageTyped filter { msg => ui.isSelectedChat(node) } }

  def messageLog(node: Remote[Node]) = on[Node] {
    ((messageSent from node).asLocal || (messageSent to node)).list }

  val chatLogs = on[Node] {
    remote[Node].joined.fold(List.empty[Signal[List[String]]]) {
      case (chats, node) => messageLog(node) :: chats } }
}

Tweets

The example shows how the operators in a processing pipeline can be placed on different peers to count the tweets that each author produces in a tweet stream. The application receives a stream of tweets on the Input peer, selects those containing the "multitier" string on the Filter peer, extracts the author for each tweet on the Mapper peer, and stores a signal with a map counting the tweets from each author on the Folder peer.

@multitier object TweetAuthoring {
  @peer type Input <: { type Tie <: Single[Filter] }
  @peer type Filter <: { type Tie <: Single[Mapper] with Single[Input] }
  @peer type Mapper <: { type Tie <: Single[Folder] with Single[Filter] } 
  @peer type Folder <: { type Tie <: Single[Mapper] }

  val tweetStream: Event[Tweet] on Input = placed { retrieveTweetStream() }

  val filtered: Event[Tweet] on Filter = placed {
    tweetStream.asLocal filter { tweet => tweet.hasHashtag("multitier") } }

  val mapped: Event[Author] on Mapper = placed {
    filtered.asLocal map { tweet => tweet.author } }

  val folded: Signal[Map[Author, Int]] on Folder = placed {
    mapped.asLocal.fold(Map.empty[Author, Int].withDefaultValue(0)) {
      (map, author) => map.updated(map(author) + 1) } }
}

E-Mail Application

The example shows a client–server e-mail application. The server stores a list of e-mails. The client can request the e-mails received in the n previous days containing a given word. The client user interface displays the e-mails broken into several pages. If the word is not in the current page, the user is informed. The definition of word of type Signal[String]onClient defines a signal carrying strings placed on the Client peer. Thanks to multitier reactives, the client-side signal inCurrentPage is defined by the composition of the local client-side signal word and the remote server-side signal filteredEmails. The latter is defined as a composition of a local signal and two remote signals.

@multitier object MailApp {
  @peer type Server <: { type Tie <: Single[Client] }
  @peer type Client <: { type Tie <: Single[Server] }

  val word: Signal[String] on Client = placed { /* GUI input */ }
  val days: Signal[Int] on Client = placed { /* GUI input */ }

  val allEmails: Local[Signal[List[Email]]] on Server = placed {
    /* e-mail collection */ }

  val filteredEmails: Signal[List[Email]] on Server = placed {
    Signal {
      allEmails() filter { email =>
        (email.date < Date.today() - days.asLocal()) &&
        (email.text contains word.asLocal()) } } }

  val inCurrentPage: Local[Boolean] on Client = placed {
    Signal { isCurrentFirstPage(word(), filteredEmails.asLocal()) }
}

Master–worker

The example shows a ScalaLoci implementation of the master–worker pattern where a master node dispatches tasks – double a number, for simplicity – to workers. The taskStream on the master carries the tasks as events. The assocs signal contains the assignments of workers to tasks. It folds over the taskStream||taskResult.​asLocalFromAllSeq event stream that fires for every new task (taskStream) and every completed task (taskResult.​asLocalFromAllSeq). The assignTasks method assigns a worker to the new task (taskAssocs), or enqueues the task if no worker is free (taskQueue) based on the folded event (taskChanged) and the currently connected worker instances (remote[Worker].​connected). The deployTask event subjectively provides every worker instance with the task it is assigned. Workers provide the result in the taskResult event stream which the master aggregates into the result signal. The signal is updated for every event to contain the sum of all values carried by the events.

@multitier object MasterWorker {
  @peer type Master <: { type Tie <: Multiple[Worker] }
  @peer type Worker <: { type Tie <: Single[Master] }

  class Task(v: Int) { def exec: Int = 2 * v }

  val taskStream: Local[Event[Task]] on Master = placed { Evt[Task] }
    // to add tasks: `taskStream.fire(Task(42))`

  val assocs: Local[Signal[Map[Remote[Worker], Task]]] on Master = placed {
    (taskStream || taskResult.asLocalFromAllSeq)
      .fold(Map.empty[Remote[Worker], Task], List.empty[Task])
       { (taskAssocs, taskQueue, taskChanged) =>
         assignTasks(taskAssocs, taskQueue, taskChanged, remote[Worker].connected) } }

  val deployTask = on[Master] sbj { worker: Remote[Worker] =>           // Event[Task]
    Signal{ assocs().get(worker) }.changed }
  val taskResult = on[Worker] {                                         // Event[Int]
    deployTask.asLocal collect { case Some(task) => task.exec } }
  val result = on[Master] {                                             // Signal[Int]
    taskResult.asLocalFromAllSeq.fold(0){ case (acc, (worker, result)) => acc + result } }
}

Publications

  1. Pascal Weisenburger and Guido Salvaneschi. 2019. Multitier Modules. In Proceedings of the 33rd European Conference on Object-Oriented Programming, ECOOP (Leibniz International Proceedings in Informatics (LIPIcs)). July 15–19, 2019, London, United Kingdom. Schloss Dagstuhl – Leibniz-Zentrum für Informatik, Dagstuhl, Germany. http://doi.org/10.4230/LIPIcs.ECOOP.2019.3
  2. Pascal Weisenburger and Guido Salvaneschi. 2019. Tutorial: Developing Distributed Systems with Multitier Programming. In Proceedings of the 13th ACM International Conference on Distributed and Event-based Systems, DEBS. June 24–28, 2019, Darmstadt, Germany. ACM, New York, NY, USA, 203–204. http://doi.org/10.1145/3328905.3332465
  3. Pascal Weisenburger, Mirko Köhler, and Guido Salvaneschi. 2018. Distributed System Development with ScalaLoci. Proceedings of the ACM on Programming Languages 2, OOPSLA, Article 129 (October 2018), 30 pages. http://doi.org/10.1145/3276499
  4. Pascal Weisenburger, Tobias Reinhard, and Guido Salvaneschi. 2018. Static Latency Tracking with Placement Types. In Companion Proceedings for the ISSTA/ECOOP 2018 Workshops (FTfJP’18). July 16–21, 2018, Amsterdam, Netherlands. ACM, New York, NY, USA, 34–36. http://doi.org/10.1145/3236454.3236486
  5. Pascal Weisenburger. 2016. Multitier Reactive Abstractions. In Companion Proceedings of the 2016 ACM SIGPLAN International Conference on Systems, Programming, Languages and Applications: Software for Humanity (SPLASH Companion 2016). October 30 – November 4, 2016, Amsterdam, Netherlands. ACM, New York, NY, USA, 18–20. http://doi.org/10.1145/2984043.2984051