Getting Started with Linear Regression
======================================

With Mlektic, you can perform univariate or multivariate linear regression using the least squares method or by considering various optimization options available in the `optimizer_archt` module.

Supported optimization methods include:
- ‘sgd-standard’
- ‘sgd-stochastic’
- ‘sgd-mini-batch’
- ‘sgd-momentum’
- ‘nesterov’
- ‘adagrad’
- ‘adadelta’
- ‘rmsprop’
- ‘adam’
- ‘adamax’
- ‘nadam’

For more details on the `optimizer_archt` module, please refer to the `optimizer_archt documentation <https://dialektico.com/mlektic/docs/mlektic.methods.html#module-mlektic.methods.optimizer_archt>`_.

You can also apply regularization to improve model generalization. The `regularizer_archt` module supports the following regularization methods:
- ‘l1’ (default)
- ‘l2’
- ‘elastic_net’

To learn more about the `regularizer_archt` module, please refer to the `regularizer_archt documentation <https://dialektico.com/mlektic/docs/mlektic.methods.html#module-mlektic.methods.optimizer_archt>`_.

For example, you can train a model using linear regression with standard gradient descent and L1 regularization with the `LinearRegressionArcht` module as follows:

.. code-block:: python

    from mlektic.linear_reg import LinearRegressionArcht
    from mlektic import preprocessing
    from mlektic import methods
    import pandas as pd
    import numpy as np

    # Generate random data.
    np.random.seed(42)
    n_samples = 100
    feature1 = np.random.rand(n_samples)
    feature2 = np.random.rand(n_samples)
    target = 3 * feature1 + 5 * feature2 + np.random.randn(n_samples) * 0.5

    # Create pandas dataframe from the data.
    df = pd.DataFrame({
        'feature1': feature1,
        'feature2': feature2,
        'target': target
    })

    # Create train and test sets.
    train_set, test_set = preprocessing.pd_dataset(df, ['feature1', 'feature2'], 'target', 0.8)

    # Define regularizer and optimizer.
    regularizer = methods.regularizer_archt('l1', lambda_value=0.01)
    optimizer = methods.optimizer_archt('sgd-standard')

    # Configure the model.
    lin_reg = LinearRegressionArcht(iterations=50, optimizer=optimizer, regularizer=regularizer)

    # Train the model.
    lin_reg.train(train_set)


.. code-block:: text

    Epoch 5, Loss: 15.191523551940918
    Epoch 10, Loss: 11.642797470092773
    Epoch 15, Loss: 9.021803855895996
    Epoch 20, Loss: 7.08500862121582
    Epoch 25, Loss: 5.652813911437988
    Epoch 30, Loss: 4.592779636383057
    Epoch 35, Loss: 3.807236909866333
    Epoch 40, Loss: 3.2241621017456055
    Epoch 45, Loss: 2.790440320968628
    Epoch 50, Loss: 2.4669017791748047


To learn more about the `LinearRegressionArcht` module, please refer to the `LinearRegressionArcht documentation <https://dialektico.com/mlektic/docs/mlektic.linear_reg.html#module-mlektic.linear_reg.linear_regression_archt>`_.

The cost evolution can be plotted with the `plot_cost <https://dialektico.com/mlektic/docs/mlektic.plot_utils.html#module-mlektic.plot_utils>`_ method:

.. code-block:: python

    from mlektic.plot_utils import plot_cost

    cost_history = lin_reg.get_cost_history()
    plot_cost(cost_history, dim=(7, 5))

.. raw:: html

    <p align="center">
      <img src="https://raw.githubusercontent.com/DanielDialektico/mlektic/main/files/plot.jpg" alt="cost plot" width="500">
    </p>

Different evaluation metrics can be obtained:

.. code-block:: python

    mse = lin_reg.eval(test_set, 'mse')
    rmse = lin_reg.eval(test_set, 'rmse')
    mae = lin_reg.eval(test_set, 'mae')
    mape = lin_reg.eval(test_set, 'mape')
    r2 = lin_reg.eval(test_set, 'r2')
    corr = lin_reg.eval(test_set, 'corr')

    print(f'MSE: {mse}')
    print(f'RMSE: {rmse}')
    print(f'MAE: {mae}')
    print(f'MAPE: {mape}')
    print(f'R²: {r2}')
    print(f'Pearson Coefficient: {corr}')


.. code-block:: text

    MSE: 2.451315402984619
    RMSE: 1.565667748451233
    MAE: 1.28024423122406
    MAPE: 38.169368743896484
    R²: 0.3232208490371704
    Pearson Coefficient: 0.9599050283432007

Print the parameters obtained by training:

.. code-block:: python

    print("Weights:", lin_reg.get_parameters())
    print("Intercept:", lin_reg.get_intercept())


.. code-block:: text

    Weights: [[1.0966599]
    [1.3730361]]
    Intercept: [1.9968479]      

And make predictions:

.. code-block:: python

    prediction = lin_reg.predict(2.0)
    print(f'Prediction: {prediction[0][0]}')

.. code-block:: text
    
    Prediction: 6.936239719390869

Finally, you can save the model parameters in a JSON format for future use:

.. code-block:: python

    lin_reg.save_model('linear_regression_model.json')