-
Notifications
You must be signed in to change notification settings - Fork 575
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Custom Entity Detection Issue in Presidio Version 2 #1305
Comments
@surajsonee can you please elaborate on how you did step 3? Was the yaml loaded and recognizers added? if you print the list of recognizers, are the ones from the yaml listed there? |
sure!
content of config.yaml:
from presidio_analyzer import Pattern, PatternRecognizer, AnalyzerEngine
import yaml
# Load custom entity rules from YAML file
with open('example_recognizers.yaml', 'r') as yaml_file:
custom_entity_rules = yaml.safe_load(yaml_file)
# Create an instance of AnalyzerEngine
analyzer = AnalyzerEngine()
# Iterate over each entity in the custom entity rules
for entity_name, entity_config in custom_entity_rules.items():
patterns = entity_config.get('patterns', [])
# Create a recognizer for each pattern defined for the entity
for pattern_config in patterns:
name = pattern_config.get('name')
regex = pattern_config.get('regex')
score = pattern_config.get('score', 0.8) # Default score
# Create a Pattern object
pattern = Pattern(name=name, regex=regex, score=score)
# Create a PatternRecognizer with the Pattern object
recognizer = PatternRecognizer(supported_entity=entity_name, patterns=[pattern])
# Add the recognizer to Presidio's recognizer registry
analyzer.registry.add_recognizer(recognizer) In this above example, example_recognizers.yaml is the YAML file containing the custom entity rules. The script reads this file, extracts the entity names and patterns, creates recognizers based on the extracted information, and adds them to Presidio's recognizer registry. Please let me know where I'm doing wrong. Thank you! |
Hi, I'm not sure what's wrong, as you seem to add the recognizers the right way. Could it be that BTW we have a method for adding recognizers from YAML: https://microsoft.github.io/presidio/analyzer/adding_recognizers/#reading-pattern-recognizers-from-yaml |
Thank you for the reference! |
Sure. if you change the default configuration in presidio/presidio-analyzer/app.py Line 40 in dee6562
To something more similar to the tutorial: yaml_file = "recognizers.yaml"
registry = RecognizerRegistry()
registry.load_predefined_recognizers()
registry.add_recognizers_from_yaml(yaml_file)
self.engine = AnalyzerEngine(registry=registry) You should be able to load the yaml based recognizers into the analyzer engine, and these would be used in each call. |
Thank you! I tried this but it didn't work. I have tried another solution which is working now. |
@surajsonee : hey. What is the other solution that you used ? thanks. |
@GautierT I have made some customizations to it so that it is running fine now. You can email me and I will share the code with you. |
ibadk6057@gmail.com |
I'm encountering an issue with custom entity detection in Presidio Version 2. Despite defining custom entity rules for medical numbers, including Aadhar and Health Insurance Claim Numbers (HICNs), Presidio does not seem to recognize them correctly.
I've followed the steps to create a YAML file (medical_numbers.yaml) containing the custom entity rules and mounted it into the Presidio container. However, when I test the detection using sample text containing these medical numbers, Presidio either returns incorrect entity types or fails to detect them altogether.
I've verified that the regex patterns in the YAML file are correct and aligned with the requirements for detecting Aadhar and HICNs. Additionally, I've ensured that the YAML file is correctly mounted into the Presidio container and configured for recognition.
Please advise on how to troubleshoot and resolve this issue with custom entity detection in Presidio Version 2.
I follow these steps:
Save the YAML file: Save the YAML file containing the custom entity rules (in this case, medical_numbers.yaml) to a location accessible by your Presidio container.
Mount the YAML file into the container: When running the Presidio container, mount the directory containing the YAML file to the appropriate configuration directory inside the container (usually /presidio/config/).
Command to run the Presidio container with mounting the custom entity configuration file:
Docker run -d
-v /path/to/medical_numbers.yaml:/presidio/config/medical_numbers.yaml
-p 8080:8080
mcr.microsoft.com/presidio-analyzer:latest
Configure Presidio to recognize the custom entity: Inside the container, ensure that Presidio is configured to load and recognize the custom entity rules from the mounted YAML file.
Testing detection: After configuring Presidio, you can test the detection of medical numbers, including Aadhar, and Health Insurance Claim Numbers, in your text data.
The text was updated successfully, but these errors were encountered: