DG-TSK

DG-TSK uses M-shaped feature and rule gates together with a point-based rule base to perform simultaneous feature selection and rule extraction.

Reference

Guangdong Xue, Jian Wang, Bingjie Zhang, Bin Yuan, Caili Dai, Double groups of gates based Takagi-Sugeno-Kang (DG-TSK) fuzzy system for simultaneous feature selection and rule extraction, Fuzzy Sets and Systems, Volume 469, 2023, 108627, ISSN 0165-0114, doi: 10.1016/j.fss.2023.108627.

Mathematical Formulation

Antecedent

DG-TSK uses standard Gaussian membership functions for each input feature:

where \(c_{r,d}\) and \(\sigma_{r,d} > 0\) are the center and spread for rule \(r\) and feature \(d\).

M-gate

The paper introduces a novel M-shaped gate function for both feature selection and rule extraction:

This function satisfies:

• \(M(\lambda) \in [0, 1]\) for real \(\lambda\);
• \(M(\lambda) = 1\) at \(\lambda = \pm 1\);
• large derivatives near the origin for faster early learning.

Antecedent gating

DG-TSK embeds feature gates in the antecedents. The gate value raises the base membership to a power, following the general gating mechanism described in section 2.2 of the paper (\(\mu^{M(\lambda)}\)):

When \(M(\lambda_d)=1\) the gate is fully open (membership unchanged); when \(M(\lambda_d)=0\) the gate is closed (\(\tilde{\mu}_{r,d}=1\), so the feature does not suppress the product T-norm).

Rule firing strengths are then computed with a product T-norm:

Rule base

The paper defines a point-based fuzzy rule base (P-FRB) where each training example initializes one rule. This strategy is justified because a richer candidate rule base helps the DG-TSK gate mechanism perform both rule extraction and feature selection simultaneously.

In highFIS, the model classes do not construct the exact per-sample P-FRB directly; instead, the closest built-in richer option is rule_base='en' via use_en_frb=True. At the estimator wrapper level, however, DG-TSK estimators support rule_base='pfrb' and pfrb_max_rules to build a sample-centered FRB from training data. That option creates Gaussian membership functions at selected training points and then uses a CoCo-style rule base over those sample-centered MFs.

The en FRB is richer than a CoCo-FRB but is not identical to the paper's per-sample P-FRB.

Consequent with rule gates

DG-TSK multiplies each rule's consequent by a rule gate:

For regression, the same gate forms a scalar gated consequent.

Output aggregation

The normalized rule weights are:

The final prediction is:

For regression, the same weighted sum applies to scalar rule outputs.

Training protocol

The paper describes DG-TSK as a single training phase in which feature gates, rule gates, and zero-order consequents are optimized together. After that phase, the learned gate structure is used to convert the model to first-order consequents and fine-tune the reduced model.

Code ↔ Paper Correspondence

Fidelity to the source article

The reproduction runs the protocol of the article and prints the result next to its Tables 2 and 3.

The estimators follow the article and the authors' reference implementation on the points below. With its defaults DGTSKClassifier reaches about 95% on Iris, against 96.8% in the article; a longer gate phase brings the selected features and rules to those of the article, at three times the cost.

Remaining differences:

• Consequents in the gate phase. The article trains the gates with first-order consequents whose constant term is set to the label of the rule's sample. highFIS trains the gates with zero-order consequents and converts to first order before fine-tuning.
• Precision. The reference code computes in double precision; highFIS uses the PyTorch default, single precision, unless torch.set_default_dtype(torch.float64) is set.

• The tables of the article against the authors' code. On the same folds and with the defaults, highFIS and the reference implementation agree, and both differ from the article in the same way:

  Dataset	Article	Reference code	highFIS
  Iris	96.8 / 2.3 / 3.1	94.3 / 2.0 / 5.8	95.0 / 2.0 / 5.8
  Wine	98.3 / 8.0 / 3.0	95.5 / 5.0 / 3.0	97.2 / 5.0 / 3.0
  Wdbc	96.2 / 5.0 / 2.2	91.6 / 2.1 / 2.1	91.2 / 2.0 / 2.5

  Accuracy in percent / selected features / extracted rules, ten stratified folds repeated twice, inputs scaled to [0, 1]. On Wine the two implementations select the same features in all 20 folds. The difference comes from the length of the gate phase, which the article does not state for its table: with dg_epochs=1000 both implementations reach the accuracy and the structure of the article (Iris 95.8 / 2.6 / 3.0 and Wine 97.2 / 9.5 / 3.0 with highFIS, ten folds repeated ten times). See the reproduction. - The regressor is an extension. The article only treats classification. DGTSKRegressor uses the same rule base, starts each rule from the target of its sample and keeps the rules that pass the threshold, which is usually one or two. On the Friedman-1 problem it matches ridge regression (R² about 0.63) while using only informative features; it has no published result to be checked against. The default 300 epochs per phase are the ones of the classifier and are short for regression: on a linear target the defaults give an R² of 0.55 and dg_epochs=3000, finetune_epochs=3000 give 0.98.

Implementation notes

• The paper's P-FRB is not implemented verbatim in highFIS. The en FRB is the closest available richer candidate rule base. When using rule_base='pfrb' via the estimator, call model_.init_consequents_from_labels(y_t) before fit_dg_phase() to apply the paper-faithful one-hot bias initialisation (paper eq. 24).
• gate_m is the default M-gate in highFIS and matches the paper's M-shaped gate function.
• DG-TSK feature gating is implemented as exponential gating: each membership value is raised to the power of its gate value (\(\mu^{M(\lambda_d)}\)), matching the general antecedent gating mechanism described in the paper.
• The DG phase freezes antecedent parameters and trains only gate params and zero-order consequents (paper §3.3). The recommended Phase 2 workflow differs by task:
  • Classification (use_lse=False): search_thresholds evaluates the zero-order model on the validation set directly, preserving the quality of the gate-training phase. Call fit_finetune afterwards to convert to first-order and retrain the pruned model. In estimator defaults, this fine-tune stage runs with antecedents enabled to improve paper conformance with the text in §3.3.
  • Regression (use_lse=True): search_thresholds prunes gates and fits first-order consequents via least squares in a single step. The LSE result is the final model — do not call fit_finetune afterwards, as it would reset the LSE-fitted weights and retrain from zero.
• fit_finetune supports both behaviours: freeze antecedents (freeze_antecedents=True, model-level default) or update antecedents (freeze_antecedents=False). DGTSKClassifier now defaults to freeze_antecedents_finetune=False for better paper conformance in the end-to-end estimator pipeline.
• Loss function: the paper uses MSE (eq. 8) throughout, including for classification (applied to one-hot targets). highFIS follows that paper contract in the DG path: classification uses MSE with one-hot targets, and regression uses MSE on scalar targets.
• DGTSKClassifier and DGTSKRegressor support both classification and regression in the same DG-TSK style.

highFIS API summary

• init_consequents_from_labels(y) — (classifier only) initialise zero-order consequent biases with one-hot encoded labels (paper eq. 24 / P-FRB). Call before fit_dg_phase().
• fit_dg_phase(x, y, **kwargs) — train DG-TSK with zero-order consequents; antecedent MF parameters are frozen (paper §3.3).
• convert_to_first_order() — convert the model to first-order consequents while preserving rule gates.
• compute_thresholds(zeta_lambda, zeta_theta) — compute pruning thresholds from gate activations.
• apply_thresholds(tau_lambda, tau_theta) — prune features and rules by zeroing gates.
• search_thresholds(x, y, *, zeta_lambda, zeta_theta, x_val, y_val, use_lse, inplace, ...) — grid-search over (zeta_lambda, zeta_theta) pairs and select the best gate thresholds by validation score. When use_lse=True, each candidate also fits first-order consequents via least squares before scoring (recommended for regression). When use_lse=False, the zero-order model is scored directly (recommended for classification). With inplace=True, the winning thresholds are applied to self.
• fit_finetune(x, y, **kwargs) — convert the pruned zero-order model to first-order, reset consequent weights to zero, and retrain. Use freeze_antecedents=True to keep MFs and λ-gates fixed, or freeze_antecedents=False to optimize antecedents and consequents jointly. Call this only after search_thresholds(use_lse=False) (classification path). Do not call it after search_thresholds(use_lse=True), which already produces final first-order consequents via LSE.

Estimator defaults (paper-conformance profile for classification): - DGTSKClassifier: rule_base='pfrb', pfrb_max_rules=300, zeta_lambda=[0.5], zeta_theta=[0.01], use_lse=False, freeze_antecedents_finetune=False, optimizer_type='sgd', learning_rate=0.2.

Usage examples

Classification

For classification, Phase 2 uses use_lse=False so that threshold candidates are ranked by the zero-order model's accuracy on the validation set. After selecting the best thresholds, fit_finetune converts the model to first-order and retrains the pruned model (antecedents can be frozen or unfrozen via freeze_antecedents).

from highfis.models import DGTSKClassifierModel
from highfis.memberships import GaussianMF

input_mfs = {
    "x1": [GaussianMF(mean=0.0, sigma=1.0), GaussianMF(mean=1.0, sigma=1.0)],
    "x2": [GaussianMF(mean=-1.0, sigma=1.0), GaussianMF(mean=1.0, sigma=1.0)],
}

model = DGTSKClassifierModel(
    input_mfs,
    n_classes=2,
    gate_fea="gate_m",
    gate_rule="gate_m",
    use_en_frb=True,
)

# Optional: P-FRB one-hot initialisation (paper eq. 24).
model.init_consequents_from_labels(y_train_t)

history = model.fit_dg_phase(X_train, y_train, epochs=30, learning_rate=1e-3)

# Phase 2a: select thresholds by evaluating the zero-order model.
result = model.search_thresholds(
    X_train,
    y_train,
    zeta_lambda=[0.5],
    zeta_theta=[0.01],
    x_val=X_val,
    y_val=y_val,
    use_lse=False,   # evaluate zero-order quality; do not fit first-order here
    inplace=True,
)
print(result)

# Phase 2b: convert to first-order and fine-tune (MFs and λ-gates frozen).
model.fit_finetune(X_train, y_train, epochs=60, learning_rate=1e-3)

Regression

For regression, Phase 2 uses use_lse=True so that each threshold candidate is evaluated after fitting first-order consequents via least squares. The best candidate's LSE-fitted model is the final result — do not call fit_finetune, which would reset those weights.

from highfis.models import DGTSKRegressorModel
from highfis.memberships import GaussianMF

input_mfs = {
    "x1": [GaussianMF(mean=0.0, sigma=1.0), GaussianMF(mean=1.0, sigma=1.0)],
    "x2": [GaussianMF(mean=-1.0, sigma=1.0), GaussianMF(mean=1.0, sigma=1.0)],
}

model = DGTSKRegressorModel(
    input_mfs,
    gate_fea="gate_m",
    gate_rule="gate_m",
    use_en_frb=True,
)

history = model.fit_dg_phase(X_train, y_train, epochs=60, learning_rate=1e-3)

# Phase 2: select thresholds and fit first-order consequents via LSE in one step.
result = model.search_thresholds(
    X_train,
    y_train,
    zeta_lambda=[0.0, 0.25, 0.5, 0.75, 1.0],
    zeta_theta=[0.0, 0.25, 0.5],
    x_val=X_val,
    y_val=y_val,
    use_lse=True,    # fit first-order via LSE; this IS the final model
    inplace=True,
)
print(result)
# No fit_finetune here — LSE consequents are already optimal.