10 questions about post-market surveillance (PMS) requirements of the MDR

Post-market surveillance (PMS) is an important part of the regulatory framework for medical devices in Europe. The Medical Device Regulation (MDR) lays special emphasis on gathering clinical and safety-related data after the approval/CE certification process and market access.

Monitoring product performance of CE-marked products is crucial to systematically identifying risks in the practical use of the product, as some risks only become apparent when medical devices are used, stored, transported or cleaned. Only through continuous and systematic post-market surveillance can manufacturers ensure that medical devices are safe and that there are no uncontrolled risks such as defects or undetected security problems.

1. What is post-market surveillance (PMS)?

The MDR (Art 2 (60)) defines post-market surveillance as a proactive and systematic process which manufacturers implement and carry out (with other economic operators) in order to take corrective and preventive action (CAPA) in accordance with information on medical devices and their performance. The surveillance and reporting of incidents involving medical devices allows identification of problems with the design, manufacture or use of medical devices and, ultimately, enhances patient safety. 

The aim of the post-market surveillance system is to actively and systematically gather, record and analyse relevant data on the quality, performance and safety of a device throughout its entire lifetime. This allows manufacturers to continuously update the risk-benefit assessment and to initiate necessary measures without delay. Manufacturers are obliged to collect and assess all information about their medical devices and related devices from competitors.

2. What is the difference between post-market surveillance and vigilance?

Vigilance is only one part of the post-market surveillance system, as it refers to the reporting of (serious) incidents, field safety corrective actions (FSCAs) and recalls. It is a reactive system that deals with incidents, rather than the proactive collection of PMS data. Section 2 of Chapter VII of the MDR on vigilance defines the incidents that manufacturers have to report to the relevant competent authorities and how to submit these reports. Also, it requires manufacturers to analyse their vigilance data.

Additional guidance in relation to the vigilance system currently in operation under the MDD (to complement the MEDDEV 2.12-1 rev 8 from 2013) was published in July 2019. This document refers to the new manufacturer incident report form, which already includes the IMDRF adverse event terminology, UDI (Unique Device Identification) and the single registration number as prescribed in the MDR. You can learn more about UDI & MDR here.

3. What are manufacturers’ post-market surveillance obligations?

The conformity assessment procedures for obtaining a CE mark, as outlined in Annexes IX to XI, require that manufacturers establish and maintain a post-market surveillance process that is proportionate to the risk class and the type of the device. This means, post-market surveillance is required regardless of the classification of the medical device, but the details of the requirements differ.

Manufacturers have to base their post-market surveillance system on a post-market surveillance plan (Article 84), which is part of the technical documentation and proves compliance with the PMS requirements of the MDR. Annex III specifies the requirements and the content of such a post-market surveillance plan, which has to address the collection and utilization of the post-market information and cover at least:

• a proactive and systematic process to collect any information referred to in point (a). The process shall allow a correct characterisation of the performance of the devices and shall also allow a comparison to be made between the device and similar products available on the market;
• effective and appropriate methods and processes to assess the collected data;
• suitable indicators and threshold values that shall be used in the continuous reassessment of the benefit-risk analysis and of the risk management as referred to in Section 3 of Annex I;
• effective and appropriate methods and tools to investigate complaints and analyse market-related experience collected in the field;
• methods and protocols to manage the events subject to the trend report as provided for in Article 88, including the methods and protocols to be used to establish any statistically significant increase in the frequency or severity of incidents as well as the observation period;
• methods and protocols to communicate effectively with competent authorities, notified bodies, economic operators and users;
• reference to procedures to fulfil the manufacturers obligations laid down in Articles 83, 84 and 86;
• systematic procedures to identify and initiate appropriate measures including corrective actions;
• effective tools to trace and identify devices for which corrective actions might be necessary;
• and a Post-Market Clinical Follow-up (PMCF) plan as referred to in Part B of Annex XIV, or a justification as to why a PMCF is not applicable.

4. What is a post-market surveillance report?

Manufacturers of Class I medical devices (incl. classes Is, Im and Ir) are required to prepare a post-market surveillance report to summarize the results and conclusions of the data gathered as defined in the PMS plan. The report includes the rationale behind and the description of preventive and corrective actions that have been taken; and it has to be updated when necessary.

5. When is a Periodic Safety Update Report (PSUR) required?

A Periodic Safety Update Report is required from manufacturers of class IIa, class IIb and class III devices. In this report, the results and conclusions of the analysis of the PMS data have to be summarized and preventive as well as corrective actions described and explained. The PSUR has to include the conclusions of the benefit-risk determination, the main findings of the post-market clinical follow-up, and the volume of sales of the device together with information on the population using the device.

The PSUR is part of the technical documentation and has to be updated regularly: at least every two years for class IIa devices and annually for class IIb and class III devices.

The PSUR of class III devices has to be submitted (through the electronic system on vigilance and post-market surveillance referred to in Article 92) to the notified body. The notified body adds its evaluation to the report and both documents will be made available to competent authorities through the electronic system. Manufacturers of implantable devices and class III devices are also required to use PMS data to update their summary of safety and clinical performance as outlined in Article 32.

Manufacturers have to make PSURs of class IIa and class IIb devices available to their notified bodies and, on request, to competent authorities.

6. What is a Post-Market Clinical Follow-up (PMCF)?

The PMCF is the systematic collection of clinical data with the aim of answering important questions about the safety or performance of the medical device and updating its clinical evaluation.

Post-market surveillance data and information has to be included in the post-market section of the Clinical Evaluation Report (CER). The process to continuously update the clinical evaluation with this data is called Post-Market Clinical Follow-up (PMCF) and outlined in Part 2 of Annex XIV. The MDR obliges manufacturers to proactively collect and evaluate clinical data from the end users of their devices to confirm the safety and performance throughout the expected lifetime of the device. This helps them ensure the acceptability of already identified risks and to detect emerging risks on the basis of factual evidence.

Manufacturers have to base their PMCF process on a PMCF plan; the findings of the PMCF have to be documented in a PMCF evaluation report, which is part of the clinical evaluation report and the technical documentation. The conclusions of the PMCF may also lead to an update of the risk management documents.

Post-market data has to be used to continually reassess the benefit–risk analysis in the CER and to update the risk management part of the technical documentation. Any regulatory action (such as recalls and notifications) has to be discussed, incidents should be displayed in a table and (serious) adverse events mentioned in detail (with a focus on the evaluation of whether an incident is device-related or not).

7. What kind of data should be collected to meet PMCF requirements?

PMCF can consist of data gathered from the vigilance system, complaints, technical information and publicly available information, and does not refer only to PMCF studies. The MDR mentions the gathering of clinical experience gained, feedback from users, screening of scientific literature as general methods and procedures of collecting relevant data. It treats the evaluation of suitable registers or PMCF studies as specific methods. The PMCF plan requires a rationale for the appropriateness of the methods and procedures a manufacturer chooses to apply.

8. What are the PMS requirements imposed by applicable QMS and risk management standards?

The ISO quality management and risk management standards also embrace the concept of PMS and Art. 83 of the MDR states that the post-market surveillance system is an integral part of the manufacturer’s quality management system.

To ensure the effectiveness of the QMS and the safety of medical devices, ISO 13485 requires a systematic post-market surveillance. Chapter 8 outlines the requirements for a continuous procedure for feedback processing which includes data from production and post-production activities. Information gathered in the feedback process has to be considered as input for risk management.

ISO 14971 requirements in the post-market phase do not focus on reporting obligations but aim to take into account PMS data to review if the probabilities and severity of possible damage are correctly estimated, the risks are fully identified and if the presumed risk acceptance criteria and benefit–risk ratios are valid.

In comparison with the MDR, the less comprehensive PMS requirements imposed by EN ISO 13485:2016 and EN ISO 14971:2012 are already obligatory, as the standards are harmonized for the Medical Device Directive.

9. What changes concerning the post-market surveillance does Eudamed introduce?

PMS requirements as well as the periodic update of the clinical evaluation already existed before the MDR came into force. But the MDR has significantly increased the post-market requirements for manufacturers and their obligation to proactively collect data and document and report the analysis of these post-market activities and data.

Greater transparency on post-market surveillance will be introduced with Eudamed’s module on vigilance and post-market surveillance and the obligation to include and update this information in the database.

10. What is the deadline for complying with MDR post-market surveillance requirements?

Post-market surveillance, vigilance and market surveillance requirements of the MDR will apply after the date of application: 26th May 2021.

How Decomplix can help

In case you are looking for post-market surveillance or other actions necessary to ensure compliance of your medical products, feel free to further explore Decomplix services here.

(last updated in April 2020)

Pictures:  Mohd KhairilX, PopTika /Shutterstock

Part 2: How does object detection work?

Now that we know a bit about what object detection is, the distinctions between different types of object detection, and what it can be used for, let’s explore in more depth how it actually works.

In this section, we’ll look at several deep learning-based approaches to object detection and assess their advantages and limitations. Just as a reminder—for the purposes of this overview, we’re going to look at the approaches that use neural networks, which have become the state-of-the-art methods for object detection.In this section, we’ll look at several deep learning-based approaches to object detection and assess their advantages and limitations. Just as a reminder—for the purposes of this overview, we’re going to look at the approaches that use neural networks, which have become the state-of-the-art methods for object detection.

Basic structure

Deep learning-based object detection models typically have two parts. An encoder takes an image as input and runs it through a series of blocks and layers that learn to extract statistical features used to locate and label objects. Outputs from the encoder are then passed to a decoder, which predicts bounding boxes and labels for each object.

The simplest decoder is a pure regressor. The regressor is connected to the output of the encoder and predicts the location and size of each bounding box directly. The output of the model is the X, Y coordinate pair for the object and its extent in the image. Though simple, this type of model is limited. You need to specify the number of boxes ahead of time. If your image has two dogs, but your model was only designed to detect a single object, one will go unlabeled. However, if you know the number of objects you need to predict in each image ahead of time, pure regressor-based models may be a good option.

An extension of the regressor approach is a region proposal network. In this decoder, the model proposes regions of an image where it believes an object might reside. The pixels belonging to these regions are then fed into a classification subnetwork to determine a label (or reject the proposal). It then runs the pixels containing those regions through a classification network. The benefit of this method is a more accurate, flexible model that can propose arbitrary numbers of regions that may contain a bounding box. The added accuracy, though, comes at the cost of computational efficiency.

Single shot detectors (SSDs) seek a middle ground. Rather than using a subnetwork to propose regions, SSDs rely on a set of predetermined regions. A grid of anchor points is laid over the input image, and at each anchor point, boxes of multiple shapes and sizes serve as regions. For each box at each anchor point, the model outputs a prediction of whether or not an object exists within the region and modifications to the box’s location and size to make it fit the object more closely. Because there are multiple boxes at each anchor point and anchor points may be close together, SSDs produce many potential detections that overlap. Post-processing must be applied to SSD outputs in order to prune away most of these predictions and pick the best one. The most popular post-processing technique is known as non-maximum suppression.

Finally, a note on accuracy. Object detectors output the location and label for each object, but how do we know how well the model is doing? For an object’s location, the most commonly-used metric is intersection-over-union (IOU). Given two bounding boxes, we compute the area of the intersection and divide by the area of the union. This value ranges from 0 (no interaction) to 1 (perfectly overlapping). For labels, a simple “percent correct” can be used.

Model architecture overview

R-CNN, Faster R-CNN, Mask R-CNN

A number of popular object detection models belong to the R-CNN family. Short for region convolutional neural network, these architectures are based on the region proposal structure discussed above. Over the years, they’ve become both more accurate and more computationally efficient. Mask R-CNN is the latest iteration, developed by researchers at Facebook, and it makes a good starting point for server-side object detection models.

YOLO, MobileNet + SSD, SqueezeDet

There are also a number of models that belong to the single shot detector family. The main difference between these variants are their encoders and the specific configuration of predetermined anchors. MobileNet + SSD models feature a MobileNet-based encoder, SqueezeDet borrows the SqueezeNet encoder, and the YOLO model features its own convolutional architecture. SSDs make great choices for models destined for mobile or embedded devices.

CenterNet

More recently, researchers have developed object detection models that do away with the need for region proposals entirely. CenterNet treats objects as single points, predicting the X, Y coordinates of an object’s center and its extent (height and width). This technique has proven both more efficient and accurate than SSD or R-CNN approaches.

How object detection works on the edge

If your use case requires that object detection work in real-time, without internet connectivity, or on private data, you might be considering running your object detection model directly on an edge device like a mobile phone or IoT board.

In those cases, you’ll need to choose specific model architectures to make sure everything runs smoothly on these lower power devices. Here are a few tips and tricks to ensure your models are ready for edge deployment:

• Prune your network to include fewer convolution blocks. Most papers use network architectures that are not constrained by compute or memory resources. This leads to networks with far more layers and parameters than are required to generate acceptable predictions.
• Add a width multiplier to your model so you can adjust the number of parameters in your network to meet your computation and memory constraints. The number of filters in a convolution layer, for example, greatly impacts the overall size of your model. Many papers and open-source implementations will treat this number as a fixed constant, but most of these models were never intended for mobile use. Adding a parameter that multiplies the base number of filters by a constant fraction allows you to modulate the model architecture to fit the constraints of your device. For some tasks, you can create much, much smaller networks that perform just as well as large ones.
• Shrink models with quantization, but beware of accuracy drops. Quantizing model weights can save a bunch of space, often reducing the size of a model by a factor of 4 or more. However, accuracy will suffer. Make sure you test quantized models rigorously to determine if they meet your needs.
• Input and output sizes can be smaller than you think! If you’re designing a photo organization app, it’s tempting to think that your object detection model needs to be able to accept full resolution photos as an input. In most cases, edge devices won’t have nearly enough processing power to handle this. Instead, it’s common to train object detection models at modest resolutions, then downscale input images at runtime.

To see just how small you can make these networks with good results, check out this post on creating a tiny object detection model for mobile devices.

Back to top