Current section

Files

Jump to
google_api_dataproc lib google_api dataproc v1 model spark_job.ex
Raw

lib/google_api/dataproc/v1/model/spark_job.ex

# Copyright 2019 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# NOTE: This class is auto generated by the elixir code generator program.
# Do not edit the class manually.
defmodule GoogleApi.Dataproc.V1.Model.SparkJob do
@moduledoc """
A Cloud Dataproc job for running Apache Spark (http://spark.apache.org/) applications on YARN.
## Attributes
* `archiveUris` (*type:* `list(String.t)`, *default:* `nil`) - Optional. HCFS URIs of archives to be extracted in the working directory of Spark drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.
* `args` (*type:* `list(String.t)`, *default:* `nil`) - Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
* `fileUris` (*type:* `list(String.t)`, *default:* `nil`) - Optional. HCFS URIs of files to be copied to the working directory of Spark drivers and distributed tasks. Useful for naively parallel tasks.
* `jarFileUris` (*type:* `list(String.t)`, *default:* `nil`) - Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Spark driver and tasks.
* `loggingConfig` (*type:* `GoogleApi.Dataproc.V1.Model.LoggingConfig.t`, *default:* `nil`) - Optional. The runtime log config for job execution.
* `mainClass` (*type:* `String.t`, *default:* `nil`) - The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in jar_file_uris.
* `mainJarFileUri` (*type:* `String.t`, *default:* `nil`) - The HCFS URI of the jar file that contains the main class.
* `properties` (*type:* `map()`, *default:* `nil`) - Optional. A mapping of property names to values, used to configure Spark. Properties that conflict with values set by the Cloud Dataproc API may be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
"""
use GoogleApi.Gax.ModelBase
@type t :: %__MODULE__{
:archiveUris => list(String.t()),
:args => list(String.t()),
:fileUris => list(String.t()),
:jarFileUris => list(String.t()),
:loggingConfig => GoogleApi.Dataproc.V1.Model.LoggingConfig.t(),
:mainClass => String.t(),
:mainJarFileUri => String.t(),
:properties => map()
}
field(:archiveUris, type: :list)
field(:args, type: :list)
field(:fileUris, type: :list)
field(:jarFileUris, type: :list)
field(:loggingConfig, as: GoogleApi.Dataproc.V1.Model.LoggingConfig)
field(:mainClass)
field(:mainJarFileUri)
field(:properties, type: :map)
end
defimpl Poison.Decoder, for: GoogleApi.Dataproc.V1.Model.SparkJob do
def decode(value, options) do
GoogleApi.Dataproc.V1.Model.SparkJob.decode(value, options)
end
end
defimpl Poison.Encoder, for: GoogleApi.Dataproc.V1.Model.SparkJob do
def encode(value, options) do
GoogleApi.Gax.ModelBase.encode(value, options)
end
end