Weka

In this section:

Description
Available Versions
Loading the Module
Classification
Regression
Clustering
Cross-Validation
Attribute Selection
Job Array for Multiple Algorithms
Hyperparameter Optimization
Results Analysis
Useful Commands
References

Description 

According to the page of Weka, Weka is a collection of machine learning algorithms for data mining tasks. It contains tools for data preparation, classification, regression, clustering, association rules, and visualization.

Available Versions 

weka/3.8.2 (default)

Loading the Module 

# Load Weka
module load weka/3.8.2

# Verify installation
weka -h

# Available variables
echo $CLASSPATH   # Path to weka.jar

Note

When loading the Weka module, the java module is automatically loaded and a weka alias is created to run the program.

Classification 

submit_weka_classify.sh

#!/bin/bash
#SBATCH -J weka_classify
#SBATCH -N 1
#SBATCH -n 1
#SBATCH -t 24:00:00
#SBATCH --mem=8G

export INPUT="train.arff test.arff"
export OUTPUT="classification_results.txt"

module load weka/3.8.2

job-nanny weka weka.classifiers.trees.RandomForest \
             -t train.arff -T test.arff -p 0 \
             -I 100 -K 0 -S 1 > classification_results.txt

Regression 

submit_weka_regression.sh

#!/bin/bash
#SBATCH -J weka_regression
#SBATCH -N 1
#SBATCH -n 1
#SBATCH -t 24:00:00
#SBATCH --mem=8G

export INPUT="data.arff"
export OUTPUT="regression_results.txt"

module load weka/3.8.2

job-nanny weka weka.classifiers.functions.SMOreg \
             -t data.arff -T data.arff -c last \
             -P 1.0E-12 -N 0 \
             -I "weka.classifiers.functions.supportVector.RegSMOImproved" \
             -L 1.0 -W 1 > regression_results.txt

Clustering 

submit_weka_cluster.sh

#!/bin/bash
#SBATCH -J weka_cluster
#SBATCH -N 1
#SBATCH -n 1
#SBATCH -t 12:00:00
#SBATCH --mem=8G

export INPUT="data.arff"
export OUTPUT="clustering_results.txt"

module load weka/3.8.2

job-nanny weka weka.clusterers.SimpleKMeans \
             -t data.arff -N 3 -A "weka.core.EuclideanDistance" \
             -I 500 -S 10 > clustering_results.txt

Cross-Validation 

submit_weka_cv.sh

#!/bin/bash
#SBATCH -J weka_cv
#SBATCH -N 1
#SBATCH -n 1
#SBATCH -t 24:00:00
#SBATCH --mem=8G

export INPUT="data.arff"
export OUTPUT="cv_results.txt"

module load weka/3.8.2

# 10-fold cross-validation
job-nanny weka weka.classifiers.trees.J48 \
             -t data.arff -x 10 > cv_results.txt

Attribute Selection 

submit_weka_attribute.sh

#!/bin/bash
#SBATCH -J weka_attribute
#SBATCH -N 1
#SBATCH -n 1
#SBATCH -t 12:00:00
#SBATCH --mem=8G

export INPUT="data.arff"
export OUTPUT="attribute_selection.txt"

module load weka/3.8.2

# Attribute selection with InfoGain
job-nanny weka weka.attributeSelection.InfoGainAttributeEval \
             -i data.arff > attribute_selection.txt

# With exhaustive search
job-nanny weka weka.attributeSelection.CfsSubsetEval \
             -s "weka.attributeSelection.BestFirst -D 1 -N 5" \
             -i data.arff > cfs_selection.txt

Job Array for Multiple Algorithms 

submit_weka_array.sh

#!/bin/bash
#SBATCH -J weka_array
#SBATCH --array=1-5
#SBATCH -N 1
#SBATCH -n 1
#SBATCH -t 24:00:00
#SBATCH --mem=8G

ALGORITHMS=(
    "weka.classifiers.trees.J48"
    "weka.classifiers.trees.RandomForest"
    "weka.classifiers.bayes.NaiveBayes"
    "weka.classifiers.functions.SMO"
    "weka.classifiers.lazy.IBk"
)

ALG=${ALGORITHMS[$SLURM_ARRAY_TASK_ID-1]}
ALG_NAME=$(basename $ALG)

export INPUT="data.arff"
export OUTPUT="results_${ALG_NAME}/"

module load weka/3.8.2

mkdir -p results_${ALG_NAME}

job-nanny weka $ALG -t data.arff -x 10 \
                  > results_${ALG_NAME}/cv_results.txt

job-nanny weka $ALG -t data.arff -T test.arff -p 0 \
                  > results_${ALG_NAME}/predictions.txt

Hyperparameter Optimization 

submit_weka_tune.sh

#!/bin/bash
#SBATCH -J weka_tune
#SBATCH --array=1-10
#SBATCH -N 1
#SBATCH -n 1
#SBATCH -t 24:00:00
#SBATCH --mem=8G

# Grid search for RandomForest
ITERATIONS=(10 50 100 150 200 250 300 350 400 450)
ITER=${ITERATIONS[$SLURM_ARRAY_TASK_ID-1]}

export INPUT="data.arff"
export OUTPUT="rf_iter_${ITER}/"

module load weka/3.8.2

mkdir -p rf_iter_${ITER}

job-nanny weka weka.classifiers.trees.RandomForest \
             -t data.arff -x 10 -I $ITER -K 0 -S 1 \
             > rf_iter_${ITER}/cv_results.txt

# Extract accuracy
grep "Correctly Classified Instances" rf_iter_${ITER}/cv_results.txt \
     >> rf_accuracy.txt

Results Analysis 

analyze_weka.sh

#!/bin/bash
#SBATCH -J analyze_weka
#SBATCH -N 1
#SBATCH -n 1
#SBATCH -t 01:00:00
#SBATCH --mem=2G

# Collect cross-validation results
echo "Algorithm,Accuracy" > summary.csv
for dir in results_*/; do
    alg=$(basename $dir | sed 's/results_//')
    acc=$(grep "Correctly Classified Instances" ${dir}cv_results.txt | \
          awk '{print $5}' | sed 's/%//')
    echo "$alg,$acc" >> summary.csv
done

# Plot with Python
cat > plot_results.py << 'EOF'
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns

# Read results
df = pd.read_csv('summary.csv')

# Sort by accuracy
df = df.sort_values('Accuracy', ascending=False)

# Plot
plt.figure(figsize=(10, 6))
sns.barplot(data=df, x='Algorithm', y='Accuracy')
plt.xticks(rotation=45, ha='right')
plt.title('Algorithm comparison - Accuracy (10-fold CV)')
plt.tight_layout()
plt.savefig('algorithm_comparison.png')
EOF

python plot_results.py

Useful Commands 

Weka Commands
Command	Description
`weka -h`	Display help
`weka weka.classifiers.trees.J48 -h`	Display options for a specific classifier
`weka weka.gui.GUIChooser`	Open graphical interface (requires ssh -X)

Note

To use the graphical interface of Weka, connect to the server with X11 forwarding:

ssh -X usuario@access.grid.unesp.br
module load weka/3.8.2
weka weka.gui.GUIChooser

References 

Documentation: https://waikato.github.io/weka-wiki/documentation/
Manual: https://ml.cms.waikato.ac.nz/weka/book.html
Wiki: https://waikato.github.io/weka-wiki/