Data Science Archives - Page 4 of 7

Kaggle Bosch competition using Apache Spark

Apache Spark for Kaggle competitions I competed in Kaggle Bosch competition to predict the failures during the production lines. As described in another post, I decided to approach this competition…

Code Data Science

Build and install Spark on Linux platform.

Build and install Spark on a Linux platform Here a short guide to build, install and configure Apache Spark on a Linux platform. You can decide which Spark release install…

Data Science Technology

Scala project under IntelliJ IDEA

Setup of a scala project using IntelliJ IDEA I suppose you have already downloaded and installed the community edition of IntelliJ IDEA from this link, that you have your java,…

Code Data Science

Spark code snippets

A growing post which gathers short pieces of code Let's suppose spark to be an opened spark session. # Open a spark session spark = SparkSession \ .builder.config(conf=conf) \ .appName("code-test").getOrCreate()…

Code Data Science kaggle

Spark and XGBoost using Scala

Spark and XGBoost using Scala language Recently XGBoost project released a package on github where it is included interface to scala, java and spark (more info at this link). I…

Data Science kaggle

PySpark for RedHat Kaggle competition

Using PySpark for RedHat Kaggle competition Redhat Kaggle competition is not so prohibitive from a computational point of view or data management. The use of Pandas and xgboost, R allows…