I have a input as http_poller, pulling data from a http endpoint
and output as a http
The configuration file is working well. But the issue i have is, since logstash runs on continuous basis it goes and insert duplicate records in output.
How can i avoid that . I need a solution wherein i can read the input only once?
My dataset is bit different, here is structure
My input data looks like this, there can ‘n’ number of records with same Name, so i cannot take Name or Area ID or PID as the unique identifier.