Packages

•  package root

  Definition Classes

  root

•  package cloudflow

  Definition Classes

  root

•  package spark

  Definition Classes

  cloudflow

•  abstract class SparkStreamletLogic extends StreamletLogic[SparkStreamletContext]

  Provides an entry-point for defining the behavior of a SparkStreamlet.

  Provides an entry-point for defining the behavior of a SparkStreamlet. Overide the method buildStreamingQueries to build the collection of StreamingQuery that needs to run as part of the business logic for the SparkStreamlet.

  Here's an example of how to provide a specialized implementation of SparkStreamletLogic as part of implementing a custom SparkStreamlet:

  // new custom `SparkStreamlet`
  object MySparkProcessor extends SparkStreamlet {
    // define inlets, outlets and shape
  
    // provide custom implementation of `SparkStreamletLogic`
    override def createLogic() = new SparkStreamletLogic {
      override def buildStreamingQueries = {
        val dataset = readStream(in)
        val outStream = dataset.select($"name").as[Simple]
        val query = writeStream(outStream, out, OutputMode.Append)
        Seq(query)
      }
    }
  }

  Definition Classes

  spark

• StreamingQueryExtensions

cloudflow.spark.SparkStreamletLogic

StreamingQueryExtensions 