Sciencemadness Discussion Board

Household products database and sites like it - Building a proper chemical constituent database

entheologist - 23-3-2017 at 06:02

I found this site which lists the chemical constituents found in thousands of household products, it lets you search by ingredient which is useful:
https://householdproducts.nlm.nih.gov/cgi-bin/household/list...

I could rip the content from that site and make a new site with a whole lot more functionality, but if I'm gonna go to all that trouble, I want to find at least 10 sites like this, preferably some EU based ones (that one is US products) so I can make a system that is useful to people from all over the world.

Heres the UK medicines site which lists the active constituents of all pharmaceutical products in the UK:
https://www.medicines.org.uk/emc/

Heres a Canadian one:
https://health-products.canada.ca/dpd-bdpp/dispatch-repartit...
^^ They only provide a search feature, but you can do broad searches to get huge lists so I can work with this.

and heres the Irish one:
http://www.medicines.ie/browseingredients.aspx

Heres the European medicines agency database:
http://www.ema.europa.eu/ema/index.jsp?curl=pages/medicines/...

Heres a pesticides database:
http://www.pesticideinfo.org/List_ChemicalsAlpha.jsp
which gives a list of compounds and links to products containing the compounds and gives their concentrations so this is useable. It only lists US products.

Heres the EU veterinary medicine database:
http://vet.eudrapharm.eu/vet/searchbyaz.do?backToList=atoz

They are the only "directly" useable sites I've found. By directly I mean I can get all the required information from the one site. In other cases I'll have to cross reference between multiple sites (which isn't really a problem). The internet is seriously lacking in this area. To scrape these sites and do useful things with them, I need either a list of products or ingredients and each product in the list has to link up with its ingredients, and each ingredient in the list has to list the products its found in. The only other requirement is knowing the concentration of the ingredient in the product is obviously essential for the info to be useful. I haven't found many sites like this at all.

I found plenty of sites with lists but don't give concentrations so they're unuseable. I found plenty of sites which have a search page, but don't let you browse the list of whats in their database so I can't use that (well, I can go through my own list of chemicals and search for each of them, so I can actually use this eventually).

An idea which may be promising is to gather up all the lists of products I can get, then search the MSDS databases for each product and extract the useful information (what ingredients they contain and in what concentration).

I'm actually working on a genomics/medicine related web project at the moment and I could integrate medicinal products into this project, so I would prefer to start with medicine databases. I only found two so far, the UK and Irish one both provide all the useful information. I found US, Canadian and Australian ones but they aren't useful.

If anyone knows any good product ingredient databases not listed here, it'd be greatly appreciated if you can share them. Also, if anyone wants to collaborate in this project you're more than welcome.

EDIT:
I found a list of drug databases for different countries:
http://pharm-infolinks.webnode.cz/drug-databases/national-dr...
and another similar list (agrochemicals it looks like)
https://www.eppo.int/PPPRODUCTS/information/information_ppp....

And this one has very good lists:
http://www.aesgp.eu/facts-figures/otc-ingredients/#by-parame...
but no detailed information. All the links I listed so far are useful, but by far the most useful one is that US product ingredient database, I'm hoping to find more sites like that for different countries.

[Edited on 23-3-2017 by entheologist]