Packages

c

cloudflow.spark.kafka

SparkStreamletContextImpl

class SparkStreamletContextImpl extends SparkStreamletContext

Linear Supertypes
SparkStreamletContext, Serializable, Serializable, Product, Equals, StreamletContext, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. SparkStreamletContextImpl
  2. SparkStreamletContext
  3. Serializable
  4. Serializable
  5. Product
  6. Equals
  7. StreamletContext
  8. AnyRef
  9. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new SparkStreamletContextImpl(streamletDefinition: StreamletDefinition, session: SparkSession, config: Config)

Type Members

  1. case class MountedPathUnavailableException extends Exception with Product with Serializable
    Definition Classes
    StreamletContext

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def checkpointDir(dirName: String): String

    Returns the absolute path to a mounted shared storage that can be used to store reliable checkpoints.

    Returns the absolute path to a mounted shared storage that can be used to store reliable checkpoints. Reliable checkpoints lets the Spark application persist its state across restarts and restart from where it last stopped.

    returns

    the absolute path to a mounted shared storage

    Definition Classes
    SparkStreamletContextImpl → SparkStreamletContext
  6. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native() @HotSpotIntrinsicCandidate()
  7. val config: Config
    Definition Classes
    SparkStreamletContextImpl → StreamletContext
  8. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  9. def findTopicForPort(port: StreamletPort): Topic
    Definition Classes
    StreamletContext
  10. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  11. def getMountedPath(volumeMount: VolumeMount): Path
    Definition Classes
    StreamletContext
  12. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  13. val maxOffsetsPerTrigger: Long
  14. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  15. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  16. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  17. def readStream[In](inPort: CodecInlet[In])(implicit encoder: Encoder[In], typeTag: scala.reflect.api.JavaUniverse.TypeTag[In]): Dataset[In]

    Stream from the underlying external storage and return a DataFrame

    Stream from the underlying external storage and return a DataFrame

    inPort

    the inlet port to read from

    returns

    the data read as Dataset[In]

    Definition Classes
    SparkStreamletContextImpl → SparkStreamletContext
  18. def runtimeBootstrapServers(topic: Topic): String
    Definition Classes
    StreamletContext
  19. val session: SparkSession
    Definition Classes
    SparkStreamletContext
  20. val storageDir: String
  21. final def streamletConfig: Config
    Definition Classes
    StreamletContext
  22. def streamletRef: String
    Definition Classes
    StreamletContext
  23. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  24. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  25. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  26. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  27. def writeStream[Out](stream: Dataset[Out], outPort: CodecOutlet[Out], outputMode: OutputMode, optionalTrigger: Option[Trigger] = None)(implicit encoder: Encoder[Out], typeTag: scala.reflect.api.JavaUniverse.TypeTag[Out]): StreamingQuery

    Start the execution of a StreamingQuery that writes the encodedStream to an external storage using the designated portOut

    Start the execution of a StreamingQuery that writes the encodedStream to an external storage using the designated portOut

    stream

    stream used to write the result of execution of the StreamingQuery

    outPort

    the port used to write the result of execution of the StreamingQuery

    outputMode

    the output mode used to write. Valid values Append, Update, Complete

    returns

    the StreamingQuery that starts executing

    Definition Classes
    SparkStreamletContextImpl → SparkStreamletContext

Deprecated Value Members

  1. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] ) @Deprecated
    Deprecated

Inherited from SparkStreamletContext

Inherited from Serializable

Inherited from Serializable

Inherited from Product

Inherited from Equals

Inherited from StreamletContext

Inherited from AnyRef

Inherited from Any

Ungrouped