skip to main content

Title: Reducing probes for quality of transmission estimation in optical networks with active learning

Estimating the quality of transmission (QoT) of a lightpath before its establishment is a critical procedure for efficient design and management of optical networks. Recently, supervised machine learning (ML) techniques for QoT estimation have been proposed as an effective alternative to well-established, yet approximated, analytic models that often require the introduction of conservative margins to compensate for model inaccuracies and uncertainties. Unfortunately, to ensure high estimation accuracy, the training set (i.e., the set of historical field data, or “samples,” required to train these supervised ML algorithms) must be very large, while in real network deployments, the number of monitored/monitorable lightpaths is limited by several practical considerations. This is especially true for lightpaths with an above-threshold bit error rate (BER) (i.e., malfunctioning or wrongly dimensioned lightpaths), which are infrequently observed during network operation. Samples with above-threshold BERs can be acquired by deploying probe lightpaths, but at the cost of increased operational expenditures and wastage of spectral resources. In this paper, we propose to useactive learningto reduce the number of probes needed for ML-based QoT estimation. We build an estimation model based on Gaussian processes, which allows iterative identification of those QoT instances that minimize estimation uncertainty. Numerical results using synthetically generated datasets show that, by using the proposed active learning approach, we can achieve the same performance of standard offline supervised ML methods, but with a remarkable reduction (at least 5% and up to 75%) in the number of training samples.

more » « less
Award ID(s):
Author(s) / Creator(s):
; ;
Publisher / Repository:
Optical Society of America
Date Published:
Journal Name:
Journal of Optical Communications and Networking
1943-0620; JOCNBB
Page Range / eLocation ID:
Article No. A38
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Machine learning (ML) is currently being investigated as an emerging technique to automate quality of transmission (QoT) estimation during lightpath deployment procedures in optical networks. Even though the potential network-resource savings enabled by ML-based QoT estimation has been confirmed in several studies, some practical limitations hinder its adoption in operational network deployments. Among these, the lack of a comprehensive training dataset is recognized as a main limiting factor, especially in the early network deployment phase. In this study, we compare the performance of two ML methodologies explicitly designed to augment small-sized training datasets, namely, active learning (AL) and domain adaptation (DA), for the estimation of the signal-to-noise ratio (SNR) of an unestablished lightpath. This comparison also allows us to provide some guidelines for the adoption of these two techniques at different life stages of a newly deployed optical network infrastructure. Results show that both AL and DA permit us, starting from limited datasets, to reach a QoT estimation capability similar to that achieved by standard supervised learning approaches working on much larger datasets. More specifically, we observe that a few dozen additional samples acquired from selected probe lightpaths already provide significant performance improvement for AL, whereas a few hundred samples gathered from an external network topology are needed in the case of DA.

    more » « less
  2. This paper proposes an evolutionary transfer learning approach (Evol-TL) for scalable quality-of-transmission (QoT) estimation in multi-domain elastic optical networks (MD-EONs). Evol-TL exploits a broker-based MD-EON architecture that enables cooperative learning between the broker plane (end-to-end) and domain-level (local) machine learning functions while securing the autonomy of each domain. We designed a genetic algorithm to optimize the neural network architectures and the sets of weights to be transferred between the source and destination tasks. We evaluated the performance of Evol-TL with three case studies considering the QoT estimation task for lightpaths with (i) different path lengths (in terms of the numbers of fiber links traversed), (ii) different modulation formats, and (iii) different device conditions (emulated by introducing different levels of wavelength-specific attenuation to the amplifiers). The results show that the proposed approach can reduce the average amount of required training data by up to13×<#comment/>while achieving an estimation accuracy above 95%.

    more » « less
  3. Optical network failure management (ONFM) is a promising application of machine learning (ML) to optical networking. Typical ML-based ONFM approaches exploit historical monitored data, retrieved in a specific domain (e.g., a link or a network), to train supervised ML models and learn failure characteristics (a signature) that will be helpful upon future failure occurrence in that domain. Unfortunately, in operational networks, data availability often constitutes a practical limitation to the deployment of ML-based ONFM solutions, due to scarce availability of labeled data comprehensively modeling all possible failure types. One could purposely inject failures to collect training data, but this is time consuming and not desirable by operators. A possible solution is transfer learning (TL), i.e., training ML models on a source domain (SD), e.g., a laboratory testbed, and then deploying trained models on a target domain (TD), e.g., an operator network, possibly fine-tuning the learned models by re-training with few TD data. Moreover, in those cases when TL re-training is not successful (e.g., due to the intrinsic difference in SD and TD), another solution is domain adaptation, which consists of combining unlabeled SD and TD data before model training. We investigate domain adaptation and TL for failure detection and failure-cause identification across different lightpaths leveraging real optical SNR data. We find that for the considered scenarios, up to 20% points of accuracy increase can be obtained with domain adaptation for failure detection, while for failure-cause identification, only combining domain adaptation with model re-training provides significant benefit, reaching 4%–5% points of accuracy increase in the considered cases.

    more » « less
  4. Efficient resource allocation and management can enhance the capacity of an optical backbone network. In this context, spectrum retuning via hitless defragmentation has been presented for elastic optical networks to enhance efficient spectrum accommodation while reducing the unused fragmented spaces in the spectrum. However, the quality of service committed in a service level agreement may be affected due to spectrum retuning. In particular, for transmission beyond the conventional C band, the presence of inter-channel stimulated Raman scattering can severely degrade the quality of the signal during defragmentation. To conquer this problem, this paper proposes, for the first time to our knowledge, a signal-quality-aware proactive defragmentation scheme for theC+Lband system. The proposed scheme prioritizes the minimization of the fragmentation index and quality of transmission (QoT) maintenance for two different defragmentation algorithms, namely, nonlinear-impairment (NLI)-aware defragmentation (NAD) and NLI-unaware defragmentation (NUD). We leverage machine learning techniques for QoT estimation of ongoing lightpaths during spectrum retuning. The optical signal-to-noise ratio of a lightpath is predicted for each choice of spectrum retuning, which helps to monitor the effect of defragmentation on the quality of ongoing lightpaths (in terms of assigned modulation format). Numerical results show that, compared to a baseline algorithm (NUD), the proposed NAD algorithm provides up to 15% capacity increment for smaller networks such as BT-UK, while for larger networks such as the 24-node USA network, a capacity benefit of 23% is achieved in terms of the number of served demands at 1% blocking.

    more » « less
  5. Abstract

    Clustering data is a challenging problem in unsupervised learning where there is no gold standard. Results depend on several factors, such as the selection of a clustering method, measures of dissimilarity, parameters, and the determination of the number of reliable groupings. Stability has become a valuable surrogate to performance and robustness that can provide insight to an investigator on the quality of a clustering, and guidance on subsequent cluster prioritization. This work develops a framework for stability measurements that is based on resampling and OB estimation. Bootstrapping methods for cluster stability can be prone to overfitting in a setting that is analogous to poor delineation of test and training sets in supervised learning. Stability that relies on OB items from a resampling overcomes these issues and does not depend on a reference clustering for comparisons. Furthermore, OB stability can provide estimates at the level of the item, cluster, and as an overall summary, which has good interpretive value. This framework is extended to develop stability estimates for determining the number of clusters (model selection) through contrasts between stability estimates on clustered data, and stability estimates of clustered reference data with no signal. These contrasts form stability profiles that can be used to identify the largest differences in stability and do not require a direct threshold on stability values, which tend to be data specific. These approaches can be implemented using the R package bootcluster that is available on the Comprehensive R Archive Network.

    more » « less