Cigna Provider References pointing to URLs

[rgiljohann]

I just started looking at the Cigna files. Based on what I am seeing they have separate files to download for every single provider reference? Is this a mistake? We cannot possibly download a separate file for every provider reference just to see their NPI/EIN.

Below is a small snippit of the provider reference in a smaller file I downloaded. In some of the larger files, there could be 18,000 provider reference json files to download for one plan. That does not make sense.

Is there a way to just download all of the NPI/EIN data from a single file, or another solution for the Cigna Provider Reference?

"provider_references":[
	{"provider_group_id":2000051,"location":"https://d25kgz5rikkq4n.cloudfront.net/cost_transparency/mrf/prov_reference_file_pmhs/reporting_month=2023-01/prov_allcatd_grp_id=2000051/2023-01_2000051.json?Expires=1991852775&Signature=GAoHGC0JmHyJm~8VWZx5PbMDNv5zpxxCvzw6SfOzOpceqi8YiaYWqlk3ThyM36Ewj7rjtLSK~a~-qO-xp1JQtRltKzTAqte1O-1tHGHY3CNa7XGbLqJKK9urrILirb-8SPZpbggDKFq6m9zdUVrAo9zWhCUnkN3fcLAJkHIgqfywRdldIsTj5GeG9a9~jq0Zzn0vJVnXq2onAL6PFsAKShY1CJKskelX53gd2pox0PgGnyWkj0TtyD6MTcXTuUgofG33vKKt53KIJgzva3IAmxDWuGyiL-q0IvnjXtUnTL11n9gcJjv1iHV131TuT4CS6Uqr1Vw~tkfV-zupGM59dw__&Key-Pair-Id=K1NVBEPVH9LWJP"}
   ,{"provider_group_id":2000056,"location":"https://d25kgz5rikkq4n.cloudfront.net/cost_transparency/mrf/prov_reference_file_pmhs/reporting_month=2023-01/prov_allcatd_grp_id=2000056/2023-01_2000056.json?Expires=1991852772&Signature=V00yuE40WPE9G7udhLXJsYg2QnFgTMPep6uy2utRsvG6vOrI~iTn~yV~QUOEaUkDDtvMGmQfaqVoP0DpnoGwYnK4Jsa-hkXCAH~BpyURFwbQGnK-Pj6he-L31QTGuvh~mF~QWtFb1I2UVOoR5gqm9MvIa1y7zXVtm9w0IFEF7Xj9m99BTqTaOhCOUWHLXZqJ2IyqDetLNaR--PkBCgW6KkmrBYhZRSesOWT0JzTgPJuUPgaGpJe3CwskrDtC3ka-Xw8n89Hjmuksq99xTPQS2isl8GNYYrdSmD3NINi~KuZsogoomzxrqe1TqpxVqu4YFDDjiWc6wSRU4d-xyqyVTg__&Key-Pair-Id=K1NVBEPVH9LWJP"}
   ,{"provider_group_id":2000050,"location":"https://d25kgz5rikkq4n.cloudfront.net/cost_transparency/mrf/prov_reference_file_pmhs/reporting_month=2023-01/prov_allcatd_grp_id=2000050/2023-01_2000050.json?Expires=1991852774&Signature=p7nAZE9m4jY0mP3zXvnz5UxYVdrS4QAY141QjyTzxxeIrq-8kQJm6Rq3GMeHy4TsMj09hX0Qo3uTic9AeCq8owifU0YH3rP2YT-gUY~gbW~M3~3dgHFP6ovaVK3E33o63kuAMUSsp05tQZ0pTPw1co-3otqDMszJvnIznmffW-dRnBj15BKAvlXr-ImQYZnX4uy9bHN-dS-JopQQbsMUkgbewYXWbgPVZux3FvyV7aMZURG1p-rjYNvprT1o-FpljjSapWudYN6B1ADe7~AKMHtmPZUewGZiLFZJrz3tlONWvEd~0Qu5MU2GWezlCMmOTHPlpMXgWm35PItULYh11A__&Key-Pair-Id=K1NVBEPVH9LWJP"}]


"in_network":[{"negotiation_arrangement":"capitation","name":"PC","billing_code_type":"CPT","billing_code_type_version":"","billing_code":"","description":"PCP CAP","negotiated_rates":[{"provider_references":[2000056,2000050,2000051],"negotiated_prices":[{"negotiated_type":"negotiated","negotiated_rate":141.80,"expiration_date":"9999-12-31","billing_class":"professional","billing_code_modifier":[],"additional_information":"","service_code":["01","02","03","04","05","06","07","08","09","10","11","12","13","14","15","16","17","18","19","20","21","22","23","24","25","26","27","28","29","30","31","32","33","34","35","36","37","38","39","40","41","42","43","44","45","46","47","48","49","50","51","52","53","54","55","56","57","58","59","60","61","62","63","64","65","66","67","68","69","70","71","72","73","74","75","76","77","78","79","80","81","82","83","84","85","86","87","88","89","90","91","92","93","94","95","96","97","98","99"]}]}],"covered_services":[{"billing_code_type":"CPT","billing_code_type_version":"","billing_code":"","description":"PCP CAP"}]}]}

`

[hchorton]

I ran into this while I was working on it as well. I found it pretty absurd, the only reason I can think that they did this was that adding all the arrays represented by the references to where they would be within the file balloons the size by quite a bit.

That said, it's to be expected that the Machine Readable Files will be pretty large, so it seems like it's placing needless extra effort on the people who will be ingesting the machine readable files to format it this way. I ended up writing a node.js script to automate a GET request to all the urls for provider references, pushing them into a map, and cross referencing that map to put the NPI/EIN arrays onto the in_network file itself. As mentioned prior it ended up increasing the size of the file by quite a bit, there might be a more elegant solution, but that's what worked for my purposes.

[shaselton-usds]

This is definitely a tradeoff scenario when thinking through the design of these files. Moving the large, repeatable chunks of JSON out of the file significantly reduced the already largish file sizes. It also allowed for building these provider networks once and referring to them many times thus cutting down on duplication.

With the table of contents file already containing links to external files, having scripts evaluate external provider references wasn't going to require much new functionality technically speaking (e.g. you have a script that is already fetching external references). @hchorton -- you're script makes sense and is pretty straightforward in the expectation on post-processing these files to whatever the consumer of the files wish.