R Interface to Apache Spark


[Up] [Top]

Documentation for package ‘sparklyr’ version 0.4

Help Pages

C E F H I J M N R S T

-- C --

compile_package_jars Compile Scala sources into a Java Archive (jar)
connection_config Read configuration values for a connection
copy_to Copy a local R data frame to Spark
copy_to.spark_connection Copy a local R data frame to Spark

-- E --

ensure Enforce Specific Structure for R Objects
ensure_scalar_boolean Enforce Specific Structure for R Objects
ensure_scalar_character Enforce Specific Structure for R Objects
ensure_scalar_double Enforce Specific Structure for R Objects
ensure_scalar_integer Enforce Specific Structure for R Objects

-- F --

find_scalac Discover the Scala Compiler
ft_binarizer Feature Transformation - Binarizer
ft_bucketizer Feature Transformation - Bucketizer
ft_discrete_cosine_transform Feature Transformation - Discrete Cosine Transform (DCT)
ft_elementwise_product Feature Transformation - ElementwiseProduct
ft_index_to_string Feature Transformation - IndexToString
ft_one_hot_encoder Feature Transformation - OneHotEncoder
ft_quantile_discretizer Feature Transformation - QuantileDiscretizer
ft_sql_transformer Feature Transformation - SQLTransformer
ft_string_indexer Feature Transformation - StringIndexer
ft_vector_assembler Feature Transformation - VectorAssembler

-- H --

hive_context Get the HiveContext associated with a connection

-- I --

invoke Execute a method on a remote Java object
invoke_new Execute a method on a remote Java object
invoke_static Execute a method on a remote Java object

-- J --

java_context Get the JavaSparkContext associated with a connection

-- M --

ml_als_factorization Spark ML - Alternating Least Squares (ALS) matrix factorization.
ml_binary_classification_eval Spark ML - Binary Classification Evaluator
ml_classification_eval Spark ML - Classification Evaluator
ml_create_dummy_variables Create Dummy Variables
ml_decision_tree Spark ML - Decision Trees
ml_generalized_linear_regression Spark ML - Generalized Linear Regression
ml_gradient_boosted_trees Spark ML - Gradient-Boosted Tree
ml_kmeans Spark ML - K-Means Clustering
ml_lda Spark ML - Latent Dirichlet Allocation
ml_linear_regression Spark ML - Linear Regression
ml_load Save / Load a Spark ML Model Fit
ml_logistic_regression Spark ML - Logistic Regression
ml_model Create an ML Model Object
ml_multilayer_perceptron Spark ML - Multilayer Perceptron
ml_naive_bayes Spark ML - Naive-Bayes
ml_one_vs_rest Spark ML - One vs Rest
ml_options Provide Options for Spark.ML Routines
ml_pca Spark ML - Principal Components Analysis
ml_prepare_dataframe Prepare a Spark DataFrame for Spark ML Routines
ml_prepare_features Pre-process the Inputs to a Spark ML Routine
ml_prepare_inputs Pre-process the Inputs to a Spark ML Routine
ml_prepare_response_features_intercept Pre-process the Inputs to a Spark ML Routine
ml_random_forest Spark ML - Random Forests
ml_save Save / Load a Spark ML Model Fit
ml_saveload Save / Load a Spark ML Model Fit
ml_survival_regression Spark ML - Survival Regression
ml_tree_feature_importance Spark ML - Feature Importance for Tree Models

-- N --

na.replace Replace Missing Values in Objects

-- R --

registered_extensions Register a package that implements a Spark extension
register_extension Register a package that implements a Spark extension

-- S --

sdf-saveload Save / Load a Spark DataFrame
sdf_copy_to Copy an Object into Spark
sdf_import Copy an Object into Spark
sdf_load_parquet Save / Load a Spark DataFrame
sdf_load_table Save / Load a Spark DataFrame
sdf_mutate Mutate a Spark DataFrame
sdf_mutate_ Mutate a Spark DataFrame
sdf_partition Partition a Spark Dataframe
sdf_predict Model Predictions with Spark DataFrames
sdf_read_column Read a Column from a Spark DataFrame
sdf_register Register a Spark DataFrame
sdf_sample Randomly Sample Rows from a Spark DataFrame
sdf_save_parquet Save / Load a Spark DataFrame
sdf_save_table Save / Load a Spark DataFrame
sdf_sort Sort a Spark DataFrame
sdf_with_unique_id Add a Unique ID Column to a Spark DataFrame
spark_available_versions Download and install various versions of Spark
spark_compilation_spec Define a Spark Compilation Specification
spark_config Read Spark Configuration
spark_connect Connect to Spark
spark_connection Get the spark_connection associated with an object
spark_context Get the SparkContext associated with a connection
spark_dataframe Get the Spark DataFrame associated with an object
spark_default_compilation_spec Default Compilation Specification for Spark Extensions
spark_dependency Define a Spark dependency
spark_disconnect Disconnect from Spark
spark_disconnect_all Disconnect from Spark
spark_install Download and install various versions of Spark
spark_installed_versions Download and install various versions of Spark
spark_install_dir Download and install various versions of Spark
spark_install_tar Download and install various versions of Spark
spark_jobj Get the spark_jobj associated with an object
spark_log Retrieves entries from the Spark log
spark_read_csv Read a CSV file into a Spark DataFrame
spark_read_json Read a JSON file into a Spark DataFrame
spark_read_parquet Read a Parquet file into a Spark DataFrame
spark_session Get the Spark Session associated with a connection
spark_uninstall Download and install various versions of Spark
spark_version Version of Spark for a connection
spark_version_from_home Version of Spark for a connection
spark_web Open the Spark web interface
spark_write_csv Write a Spark DataFrame to a CSV
spark_write_json Write a Spark DataFrame to a JSON file
spark_write_parquet Write a Spark DataFrame to a Parquet file

-- T --

tbl_cache Load a table into memory
tbl_uncache Unload table from memory