Wikipedia talk:Identifying reliable sources (medicine)/Archive 9

src: upload.wikimedia.org

Video Wikipedia talk:Identifying reliable sources (medicine)/Archive 9

Popular books, David Healy, and Suicide

I'm going to stick my neck out and ask for the community's opinion on what may be a controversial question: When are books reliable medical sources? I would guess that no one has a problem with Harrison's Internal Medicine and other textbooks. But we have a lot of books quoted in articles that are written by scientists and physicians acting as advocates for social change and critics of the status quo. Are these sources MEDRS or are they considered the work of one person advocacy groups? There is a remarkable spectrum out there in terms of "mainstreamness":

• Sanjay Gupta
• Ben Goldacre
• David Healy
• Dr. Mercola (yes, I think he actually has a medical license)

While this is a general question, the immediate issue I'm thinking about today is David Healy's books being a source for a tie between antidepressants and violence against others. I ran through Pubmed today, and found only a single, 8 year old MEDRS compliant reference (a review article) suggesting even the possibility of a causative role for anti-depressants in violence. There were 2 or 3 other old reviews / meta analyses suggesting an effect of SSRIs in reducing violence, and half a dozen or primary references describing clinical trials in which an anti-violence effect was observed. Its quite possible I missed a few, but this clearly is not what psychiatrists spend a lot of time worrying about in the overall scheme of things. By comparison, I got 224 hits searching for secondary references tying antidepressants to seizures, and 859 for suicide.

Overwhelmingly the psychiatric literature seems to treat the proposed relationship between antidepressants and violence as being similar to the relationship between fish and bicycles: one that does not even merit discussion. There is no mention of anti-depressant induced violence in the multi-hundred page NICE guidance on the treatment of depression, or in any other treatment guidelines written by professional medical societies that I have seen.

My personal belief (I'm sure others will disagree) is that the huge disconnect between the medical literature and popular belief (typing "antidepressants and violence" into Google gives quite a few more results than a similar search in Pubmed) is analogous to the disconnect between popular belief and expert opinion regarding vaccines and autism. We shouldn't be legitimizing a connection that is not supported (apparently not even a little bit) by mainstream medical thought.

I'm proposing to move the "violence" level 2 heading in the Prozac article out of "Adverse effects" and into "Society and Culture". I'm expecting some criticism form David Healy fans if I do this, and would like to have a measure of whether there is support here for this before doing so.

Thanks. Formerly 98 (talk) 23:06, 26 May 2014 (UTC)

Agreed. Tertiary sources that are individually Individually authored books advocating some position, and not summarizing the scientific consensus, are primary sources. David Healy in particular is WP:FRINGE. After led the charge about a supposed tie between SSRIs and suicides in kids and prescriptions dropped, the rate of suicide went up because kids were no longer getting the best drugs. See here and especially here. I just boldly edited MEDRS accordingly. Jytdog (talk) 00:29, 27 May 2014 (UTC) (struck off-target remarks. Jytdog (talk) 02:43, 27 May 2014 (UTC)) (edit so it is not nonsense. Jytdog (talk) 02:44, 27 May 2014 (UTC))

All the same, I'd prefer to keep this a discussion on the use of such books as medical sources than get into a dicussion about issues that are covere din the medical literature. Formerly 98 (talk) 02:30, 27 May 2014 (UTC)

you are right. struck. Jytdog (talk) 02:43, 27 May 2014 (UTC)

@WhatamIdoing: per your reversion of @Jytdog:'s edit of the guideline, please join the conversation! In your edit summary you said "you could have a lay book that is both advocating for a position and summarizing scientific consensus" But isn't it true that almost any source, whether an advocacy website, a primary research reference, a magazine article, or a blog "could" be summarizing scientific consensus? Isn't the purpose of MEDRS to select those that are "with very high likelihood" summarizing scientific consensus? I would think a lot of what we rule out in MEDRS is based on reduced probability that the sources is objectively summarizing concensus, and advocacy is a red flag. Your thoughts? Thanks Formerly 98 (talk) 16:56, 27 May 2014 (UTC)

What the change set up was defining all lay-oriented books advocating a position as being primary while simultaneously defining all lay-oriented books that summarized scientific consensus as being tertiary. This isn't logically possible, because you could have a lay-oriented book that does both (e.g., probably anything written by Ben Goldacre). Furthermore, you could have a lay-oriented book that is actually a secondary source. There's nothing about writing in plain English with patients as your audience that prevents you from analyzing the primary literature. There's also nothing about writing a secondary source that means your analysis is any good: crackpots can write crackpot books about crackpot ideas, and still end up with a secondary source.

The purpose of MEDRS is not to produce WP:SPOV articles. The community rejected that. The purpose of MEDRS is to help you figure out which sources represent the majority and minority (and tiny minority) viewpoints, so that you can produce a WP:DUE summary of the subject. In the instant case, that section is going to end up reading something like "David Healy says that SSRIs cause violence and suicide, but <long list of authorities> say that he's wrong". WhatamIdoing (talk) 20:03, 27 May 2014 (UTC)

I appreciate the revert of my bold edit, WAID, and what you say above. While i think the silence of MEDRS on books speaks volumes with respect to generally not using them, I think it would be helpful to say something explicit and general, if possible, about science books that are not textbooks - some guidance on how to think about them as sources. Do you think such guidance would improve MEDRS and if so, do you have suggestions? thanks. Jytdog (talk) 20:16, 27 May 2014 (UTC)

I have two immediate thoughts: One, that "textbook" is too narrow. You don't want to exclude professional reference works. Two, sometimes these are really good sources. In particular, if you want your references list to contain something that any reader could use as a source for more information (and not just those who can read medical literature), then including a reputable lay-oriented science book might be ideal. (You wouldn't want to use it to debunk or contradict the scientific literature, of course.) WhatamIdoing (talk) 21:45, 27 May 2014 (UTC)

I sort of see your point, but where I see the problem is that by your reasoning, anyone who writes a book is an authority who can be quoted to bring a point of view into an article, even if they are a minority of one among 10,000 experts. Where do you draw the line?

Let's leave aside Dr. Healy for a moment, and look at a few more cases. How about Dr. Mercola? Should we quote his opinion on vaccines and then argue it away with the 10,000 studies showing what the mainstream opinion is? How about Dr. Richardson's book on treating your cancer with laetrile instead of conventional medicine? How about Dr. Jensen's book on preventing disease by colonic irrigation?

When we discuss the value of antidepressants in mild depression or the value of statins in primary prevention, we have good quality secondary studies that have been published in peer-reviewed venues and significant numbers of qualified experts on both sides of the issue. So we present both sides. When we have one person with an extreme viewpoint that cannot be found in the peer reviewed secondary sources, I don't think we want to go there.

Anyone can write a book. Unlike scientific journals, they are not peer-reviewed. Since they are not aimed to a discerning professional audience, garbage can be published without any real risk to the publisher. A journal doing that will lose readers. Formerly 98 (talk) 20:30, 27 May 2014 (UTC)

Yes, anyone can write a book. Some of them can even manage to get them published (rather than self-published, which, by the way, is the first grounds for rejecting almost any individual's website as a reliable source). A properly published source, from an outfit with a reputation for fact-checking, is pretty much the garden-variety definition of reliable around here, even for medicine-related subjects.

But the mere existence of a book doesn't mean that you have to cite it. For broad subject areas like depression or cancer, there are thousands of reliable sources (not just books) out there, and you look at all of the sources to determine what's DUE. If an author is really one voice out of ten thousand sources, then you invoke DUE's prohibition on including the tiny-minority viewpoint at all. If it's a "regular-size" minority POV--i.e., there are at least a few people who hold this POV, and/or their POV is acknowledged (as existing, not as being correct) by other sources--then you include it, but in a way that indicates that it is the minority viewpoint. That way, readers (e.g., readers trying to learn more about the FDA-required black box label on their pill bottles) will know that people have been worried about SSRIs causing violence and suicide, and here's the currently accepted view. WhatamIdoing (talk) 21:45, 27 May 2014 (UTC)

I follow you on the suicide, but Healy is the only one out there ringing the bell on violence. (And by the way, there is no black box warning for violence on SSRI labels, only suicide). He's as fringe on that issue as Mercola is on vaccines if you compare his view to what is in the peer-reviewed literature. (Actually more so, you can find more peer-reviewed secondary references supporting the vaccine-autism link than you can find supporting the SSRI-violence link.)

There's a place for ultra minority views - In the society and culture section. Formerly 98 (talk) 23:10, 27 May 2014 (UTC)

IMO "ultra minority views" belong on their own websites. WhatamIdoing (talk) 01:03, 28 May 2014 (UTC)

Maps Wikipedia talk:Identifying reliable sources (medicine)/Archive 9

MEDLINE versus other databases

Hello everyone. This is a conversation that started on Talk:Electronic cigarette (permalink) that I am bringing over to this venue because it covers a much broader scope. I have gone back and read the history of this issue; the most relevant discussion seems to be here. The current text of the guideline is:

An integral part of finding high quality sources is avoiding articles from journals without a reputation for fact-checking and accuracy. A red flag that a journal article is probably not reliable for health claims includes publication by a publisher that has a reputation for exhibiting "predatory" behavior, which include questionable business practices and/or peer-review processes that raises concerns about the reliability of their journal articles. (See "References" section below for examples of such publishers.[14][15]) Other indications that a journal article may not be reliable are its publication in a journal that is not indexed in the bibliographic database MEDLINE,[16] or its content being outside the journal's normal scope (for instance, an article on the efficacy of a new cancer treatment in a psychiatric journal or the surgical techniques for hip replacement in a urology journal). Determining the reliability of any individual journal article may also take into account whether the article has garnered significant positive citations in sources of undisputed reliability, suggesting wider acceptance in the medical literature despite any red flags suggested here.

MEDLINE is a fantastic database, and I rely on PubMed literally every day. It covers a vast variety of journals, certainly more than I ever need to use to write a normal research article or in most cases, to write a medical article for Wikipedia. But while it covers thousands of journals (some of questionable quality), MEDLINE does not pretend to be a comprehensive database. There are dozens of others; just to name a few: EMBASE, Biological Abstracts, PsychINFO, EBSCO Academic Search, the Web of Science Science Citation Index, and the Cochrane Central Register of Controlled Trials. There is much less overlap than I would have imagined, and it isn't because the other databases include a lot of junk. There a 1/3 overlap between EMBASE and MEDLINE, and that is largely because "[t]he EMBASE data provide better coverage of European journals". Now, I'll be the first to agree that exclusion from MEDLINE makes me at least suspect a journal's reliability. But this sentence shouldn't be read as a shield against inclusion of particular journals for that reason only, and I recommend that we alter the wording somewhat.

Pinging User:Yobol (who I originally started this conversation with) and leaving a note on WT:MED. NW (Talk) 14:49, 23 May 2014 (UTC)

This sounds like a familiar topic, (must check archives...) but anyhow, "comprehensive" is the opposite of "selective". Inclusion in a "comprehensive" index such as EBSCO or EMBASE tells us nothing about the value of the content, only how to go about accessing it. Inclusion in a "selective" index tells us that it nominally meets the inclusion criteria of that index. Medline indexing is a very low bar, and it includes many serials which are very questionable. If a journal cannot even meet that standard, alarm bells should be going off. Of course, even low quality serials will sometimes include good papers, but we need some kind of evidence that someone qualified to assess the work thought that it was trustworthy, and that the someone who did so was not inside a walled garden, such as we frequently see in fringe topic areas, which often run their own journals by and for true believers. LeadSongDog come howl! 15:24, 23 May 2014 (UTC)

I think the problem is the gap between "cannot meet" and "did not choose to apply". WhatamIdoing (talk) 20:50, 23 May 2014 (UTC)

Medline inclusion criteria are here. I'd be interested to see the inclusion criteria for other databases, or if they are even explicitly written out somewhere. Formerly 98 (talk) 15:30, 23 May 2014 (UTC)

It is important to keep the scope of PubMed in mind. By definition, PubMed's coverage of reliable main stream biomedical journals is excellent, but there are closely related disciplines such as psychology where the coverage of high quality peer reviewed journal is less extensive. Boghog (talk) 20:57, 23 May 2014 (UTC)

That's a fair response, but would be nice if there was something similar in terms of being to some extent a "curated" database of sources.Formerly 98 (talk) 21:04, 23 May 2014 (UTC)

+----------------------------------------------------------------------------------------------------+ Comment. I suggest that you format this as an RfC with two outcomes: retain or remove, so that (1) more users may contribute and (2) as it is a perennial issue, this statement may be retained or removed. --LT910001 (talk) 21:06, 23 May 2014 (UTC)

Perhaps we should be clearer, e.g., its publication in a biomedical journal (assuming it's kept, which I expect that it would be. WhatamIdoing (talk) 01:41, 24 May 2014 (UTC)

@@NuclearWarfare:: I agree with what LeadSongDog said above, and note that the lack of comprehensiveness of MEDLINE indexing is a feature, not a bug. The purpose of the paragraph in question is to weed out the bad journals, which are proliferating rapidly. That MEDLINE has a clear, independent criteria to specifically do that for us is what makes it valuable. While I trust most WP:MED editors to be able to look at an article/journal to see what is adequate for our use, we can't expect that from new editors with no experience with medical literature. This problem becomes significantly worse with POV/fringe pushers who insist their garbage "review" in garbage journal needs to be added to the article. I would be happy to see what ways we can improve wording or if another criteria can be used, but MEDLINE indexing has several benefits 1) independent criteria for curating journals 2) MEDLINE indexing is a fairly low bar, with a bunch of crap journals indexed, so it is not an unnecessarily restrictive criteria 3) it is a widely used index so has a good reputation in the medical literature in general. What would be your suggestion on improving the wording that would still keep the core concept of finding a filter for bad journals? Yobol (talk) 15:00, 24 May 2014 (UTC)

agreed with Yobol. for sure. Not every journal that hangs a shingle is reliable. Jytdog (talk) 16:23, 24 May 2014 (UTC)

I disagree with Yobol. If Medline is a low bar and indexes a lot of crap it certainly isn't an unnecessarily restrictive criteria, but for the same reasons it also isn't a worthwhile one. If there are also high-quality papers that aren't Medline-indexed it just becomes even more pointless.--FergusM1970^{Let's play Freckles} 18:22, 24 May 2014 (UTC)

@NuclearWarfare: Agree with Yobol. That is exactly what Medline is and why we use it. SW3 5DL (talk) 00:44, 25 May 2014 (UTC)

• Agree with current wording regarding MEDRS per the reasons Yobol and others have given. Selectivity is a pro, not a con. Zad68 02:23, 25 May 2014 (UTC)

+----------------------------------------------------------------------------------------------------+ Hi there. Sorry to everyone for taking so long to respond. Ironically, it's because I've been doing quite a bit of cursing at EMBASE. Specifically, I need to track down some articles that I found in it from the 1980-90s that are not listed in PubMed. And I think those articles provide an excellent starting place for our discussion. I'm not going to go into the specifics of these articles, partly for my own privacy and partly because it's not fully relevant to the conversation at hand. But take it on my word that while they are rather old, they are otherwise reliable. The authors are largely well-respected faculty who have published in the well-known biomedical journals (NEJM, Lancet, Journal of <Speciality>, etc.), but who have happened to have these findings in a non-MEDLINE indexed journal. The study design is robust (they meet, as best articles articles from that era can, CONSORT / STROBE / PRISMA guidelines [see http://www.equator-network.org/ EQUATOR]). With the exception of their agedness, they would easily be eligible to be cited on Wikipedia if they had been published in a MEDLINE-indexed journal. As to why they weren't or why that journal isn't today? Beats me. But there are plenty of articles today in such journals that would be totally adequete to cite on Wikipedia in my opinion.

Now I'm not saying that this is true for all non-MEDLINE indexed journals for sure. There are a lot of terrible articles and journals out there, and I am firmly of the belief that P(Bad quality|MEDLINE)<P(Bad quality|not-MEDLINE). But that latter probability is not equivalent to 1.0, and we shouldn't automatically treat it as such (as I have seen done more than once). NW (Talk) 19:07, 30 May 2014 (UTC)

If your argument is that there are exists excellent articles in non-MEDLINE indexed journals and that there are bad articles in MEDLINE-indexed journals, I don't think you will get much of an argument from anyone. No one is saying that using MEDLINE indexing as a filter is a perfect system. However, I think some filter is necessary given the proliferation of truly bad journals, and for the reasons I stated above, MEDLINE indexing seems a reasonable starting point. If you have another system that would be an improvement over using MEDLINE indexing, I think we are all ears.

(As an aside, the paragraph in questions has an "out" for any article red-flagged, including in non-MEDLINE indexed journals - if they are positively cited by others in the medical literature, that is a sign it would be reliable, regardless of any red flags. I wonder if that applies to the articles you mention.) Yobol (talk) 19:20, 30 May 2014 (UTC)

(edit conflict) sounds like the biggest issues are that the papers you are discussing are a) primary sources and b) very old. my sense is that it will be difficult to make reasonable arguments that an article with those strikes against it and the lesser one (non--MEDLINE indexed) is great to use, especially if the content that you want to support it with is; a) a statement about reality today (as opposed to some historical context); and b) at all controversial. i am ~guessing~ it is both a statement about reality and controversial, or you would not be pushing so hard for the source, but i would be happy to proven wrong about that (generally if somebody is pushing on a shaky source it is because the content is important to them and cannot be supported otherwise......). hard to understand why work by important investigators would not be discussed in a more contemporary secondary source.... hm. hard to discuss without more detail. Jytdog (talk) 19:29, 30 May 2014 (UTC)

Sorry, I knew as soon as I posted that using examples from the 80s and 90s would confuse the issue. It's not that these particular articles aren't MEDLINE-indexed. It's that the journals they are published in do a fine job curating submissions, and they have continued to do this from the 80s through today. I'll respond to the rest of the point when I get a free moment later. NW (Talk) 20:02, 30 May 2014 (UTC)

I think you all are on the wrong track, with your assumption that NW wants to cite these old sources. He's only pointing out that there are limitations to our advice, and that we might not be explaining our real practice very well.

We say, "Other indications that a biomedical journal article may not be reliable are its publication in a journal that is not indexed in the bibliographic database MEDLINE..."

This is certainly imperfect, because it would make a bit more sense to say that "an article" gets published "in a biomedical journal that is not indexed". NW's point is that some biomedical journals may be fine, but not (currently) indexed by MEDLINE for one reason or another, including (a) being a new journal and (b) not choosing to apply, which I suspect happens more often with non-English/non-Western journals. There's also the POV problem of including "doctors" but not nurses, psychologists, allied health, vet med, and other disciplines, which would suggest that indexes like CINAHL and PsychINFO might be similarly important. The PASCAL (French; see [1]) looks like an excellent database for finding information related to safety, cognition, and environmental health. Even within the "doctoring" field, it might be appropriate to accept indexing by databases like EBMR (Ovid's Evidence-Based Medicine Reviews) or TOXLINE, if those happened to contain journals that MEDLINE didn't. MEDLINE might be good, but it's not necessarily the only suitable database out there. WhatamIdoing (talk) 01:18, 1 June 2014 (UTC)

I would be inclined to suggest that whether or not a journal is MEDLINE-indexed is one criterion that we should (and do) use to evaluate the reliability of a given source for a given purpose; it is not - and should not - be an absolute requirement in all cases. Potential exceptions may include

• some very new journals (for which we have additional supporting reasons to believe may be of high quality, and are likely to become indexed),
• older titles that have ceased publication prior to the ascendency of MEDLINE (though we may with to support such sources with more recent secondary articles that refer back to them),
• journals which may contain valid health information but which don't necessarily fit well within MEDLINE's typical scope (especially those journals which appear in other curated indexes),
• articles that are highly cited by other reliable sources,
• and probably some others that I can't think of off the top of my head.

I read the phrase in our current guideline as using the particular wording may not be instead of is not as a deliberate choice: "Other indications that a journal article may not be reliable are its publication in a journal that is not indexed in the bibliographic database MEDLINE". The intent is not to exclude all non-MEDLINE journals, but to emphasize that such sources require extra scrutiny before we use them. It's a speed bump, not a wall. TenOfAllTrades(talk) 02:38, 1 June 2014 (UTC)

I think that a "speed bump" was the intention, but it is unfortunately not the way some editors talk about the advice on this page.

What do you think on naming other suitable indexes? Or a note about some reasons when an exception to this should be strongly considered? WhatamIdoing (talk) 05:04, 1 June 2014 (UTC)

I think that, before we get into trying to rewrite this guideline, it would be a good idea to collect some examples of good, reliable sources that aren't MEDLINE-indexed, but which should be used in Wikipedia articles. In other words, are there exceptions that we apply (or should apply) broadly enough to warrant specific mention? 13:18, 1 June 2014 (UTC)

src: upload.wikimedia.org

Systematic Reviews

This open-access journal is fairly new, but looks to be useful. Comments? LeadSongDog come howl! 01:26, 14 June 2014 (UTC)

Looks interesting. I find the current trend of non anonymous referees a little odd as anonymity was originally intended to protect the integrity of the process, but this journal isn't alone in this practice.Formerly 98 (talk) 01:37, 14 June 2014 (UTC)

src: upload.wikimedia.org

Clarification

I have added "counter" to the list of things primary sources should not be used for in this edit [2]. Thoughts? Doc James (talk · contribs · email) (if I write on your page reply on mine) 21:37, 18 June 2014 (UTC)

agree Formerly 98 (talk) 22:24, 18 June 2014 (UTC)

src: upload.wikimedia.org

News reports of lab grown organs or other medical firsts

Articles like Vaginal transplantation, Trachea, Hand transplantation and Face transplant contain references from all kind of newspapers and sites like BBC, CNN, USA Today, The Guardian, Xinhua, The Daily Telegraph, Fox News, ABC News, Yahoo! and so on. Those newspaper articles report important milestones for the history of such achievements. Examples: [3] [4], [5].

My question is: What kind of sources are those? Primary or secondary or tertiary sources? What makes such news notable and reliable sources?

Thanks. --  Ark25  (talk) 11:18, 21 June 2014 (UTC)

Those are terrible sources for those claims. Those newspaper articles are indeed secondary sources, but they are not reliable secondary sources for health-related content. Quite often those articles in the press are based on press releases and light reporting following up on them, and don't carefully consider things - claims about "first" this or that in the media are often wrong. I remove them from articles I watch. Jytdog (talk) 11:30, 21 June 2014 (UTC)

For example, a clinical trial of artificial bladders "grown" from autologous cells was published in 2006 - the longest-tracked patient had been followed for 60 months so that transplant happened at least in 2001. Your third source above - the BBC article from 2011 - says in boldface that "Surgeons in Sweden have carried out the world's first synthetic organ transplant." Just dead wrong. Jytdog (talk) 11:39, 21 June 2014 (UTC)

Well, the 2001 transplant was experimental (the patient probably signed some agreement to accept the risks) and maybe the one in 2011 was the first carried out for an ordinary patient? There can be many different and significant artificial bladder firsts, and they are all important milestones.

Still, there are plenty of such references used in Wikipedia articles and it doesn't seem that we can find better references to replace them with. Do you remove just the references, or you remove the information they reference too? Or you remove such references and replace them with better ones? Do you watch articles that contain medical firsts? (firsts like: first hearth transplant, first bionic limb and so on). That kind of news are reported in generalist newspapers and websites. I'm not sure when we can expect a peer-review scientific journals to publish such firsts. They are more likely to publish successful experiments firsts than the first operations for regular patients, I think. We are not talking about medical research here. We are talking about medical firsts, based on a lot of previous research and experiments. --  Ark25  (talk) 11:51, 21 June 2014 (UTC)

I give you an example that is dead on and you argue with me? The 2006 study was better than the Swedish study - it was on 5 or six patients, not just one. It is almost as though you haven't read MEDRS - see especially the section called "Respect secondary sources" which deals explicitly with media coverage of health-related science. As that section says reminds folks, we are an encyclopedia - we are not a newspaper and we never want to fall prey to WP:RECENTISM. And of course we can find better references to replace those media articles with. Reviews in the biomedical literature very often recount the history of the fields they are reviewing. They are not easy to find with google but they are easy to find if you look in the right places - MEDRS describes where to look. I'll step back and let other experienced editors weigh in - I am sure you will hear the same from them. Jytdog (talk) 12:14, 21 June 2014 (UTC)

+----------------------------------------------------------------------------------------------------+ Sorry I didn't meant to make you angry. You are right, the BBC's article from 2011 is talking about windpipe, and instead of saying first synthetic windpipe transplant, it says first synthetic organ transplant. My point is that the BBC news presents an achievement that is most likely an important milestone - even though the article uses a wrong word. Until we find a better source, I think it's best to leave the information in the article (which the wrong word replaced) along with the reference, instead of completely removing both of them. Speaking of RECENTISM - my point is not to focus on recent news, but to present important medical achievements (not studies, not experiments and not research), even if we have to use references from BBC or USA Today. Later, another editor will replace those references with better ones. I think a 3 year old news is not exactly recent, also.

My focus is not on experiments, studies or research (you said Swedish study but it's not a study). My focus is on medical firsts, operations done to regular patients. Those operations are carried out after a lot of previous research and experiments (many of them being firsts). There is a first successful artificial urinary bladder experiment on animals, then the first such experiment on humans, and then there is the first successful operation carried out on a regular patient. And probably there are other first artificial urinary bladder firsts (the first done on a child for example). All of them are important, in my view. --  Ark25  (talk) 15:00, 21 June 2014 (UTC)

It looks like the whole of this guide (WP:MEDRS) is focused on how to treat research, studies and experiments, which is not the kind of information I am asking about. --  Ark25  (talk) 15:05, 21 June 2014 (UTC)

You have not made me angry but it is weird that you come asking questions and then argue with a direct, clear answer instead of asking more questions. Ark the report on the work in Sweden was indeed the report of an experiment. We don't know how long that tissue will last in the patient or what it will do long term, or if we will be able to do that again, many times, in many different people, safely and effectively. Sure it is an achievement, but it is not medicine or even confirmed science yet (the essence of science is repeatability). It is hard for many people to understand why MEDRS exists -- what makes health related content different from and harder to deal with than other content on Wikipedia. I am working on a little essay about this, but I recently had a dialogue with another editor on my Talk page about this. It is very rough, but for what it is worth, it is the big block of text in this section. Perhaps that will help you. I didn't point you to this yet so I will now. See Wikipedia:Identifying_reliable_sources_(medicine)#Popular_press. The popular press can have a role in health related articles, as described there. But again, I would not consider it reliable for anything hype-y like "first" of anything. Where the cited section says "historical" think about topics like the discovery of how cholera is transmitted, which the NY Times had a nice piece on recently. And as I mentioned, review articles published in the biomedical journals do discuss the history of the field, noting the actual "firsts" - they are the reliable sources for things like that. Jytdog (talk) 15:22, 21 June 2014 (UTC)

It'd be worth reading WP:PRIMARYNEWS, too. Most newspaper stories, especially of the 'this new thing just happened' or 'this new paper was published', are primary sources.

"First" is probably one of the things that a newspaper is okay for. "First" is a statement of history. It is not a statement of biomedical information. "First person to jump off a skyscraper with a parachute" and "First person to do a new surgery" ultimately need the same kinds of sources. WhatamIdoing (talk) 22:08, 21 June 2014 (UTC)

+----------------------------------------------------------------------------------------------------+ I hope I don't become a pain in the neck with so many questions. I just want to understand how things work.

I agree, a first hospital operation of its kind (and the second, and the third), although is based on lots and lots of previous research, is still an experiment, because we don't know how it will end up. But it's not a laboratory experiment. It's the first step (or a first attempt) into mainstream, and it's very important milestone.

This guide focuses on studies, experiments and research and that's how it should be, since more than 99% of the information in the medical topics is about the knowledge we extract from experiments. But in my case, I am interested about the history of medical achievements in hospital, not even about the history of any medical topic. And I don't even care which one was the very first. I am interested about the first operations of the same kind, let's say the first 5 or 10 such operations. The information I'm asking about is not about scientific facts (like the correct formula is P=U*I, or like long-chain fatty acids cannot cross the blood-brain barrier) where we need to use secondary sources, but about the historic milestones - basic facts of reality, which are most of the time reported by newspapers. While newspaper articles are much more inclined to make errors (the above article says organ instead of trachea) than academic papers, they can be very useful for having an idea where those milestones are.

Hand transplantation has a long list of such achievements, using newspaper references. When we'll have secondary sources, we will better know which one was the very first, the second, and where the other milestones are and we can adjust the Wikipedia article accordingly. But until we can find secondary sources to confirm or to contradict the information in that list, is it ok to use newspapers to add such information? --  Ark25  (talk) 09:05, 22 June 2014 (UTC)

The windpipe thing is clinical research, which is a necessary step in developing treatments. It is definitely more developed than bench research, but is still research. And again I don't know how many times I need to say this, but the press gets firsts wrong all the time with regard to medical "firsts" being reported as they happen. Even Carl Zimmer blew it just the other week when he talked about a "first" in next-gen sequencing with respect to its use to identify an infectious agent - he stated it too broadly. The popular press may be accurate for truly historical firsts (again, like the identification of how cholera spreads, which was a "first") And again and most importantly, review articles in the biomedical literature are way way more reliable than the popular media for this - why are you not responding to this? The only reasons I can see to push for using the popular press instead of review articles, as you are advocating, are: 1) it is easier than using review articles since you can find them via google and 2) WP:RECENTISM. Neither is a good justification. Jytdog (talk) 09:42, 22 June 2014 (UTC)

Maybe I'm slow in understanding certain things but it's not my purpose to irritate anyone. For me it's not important if the 2011 synthetic windpipe transplant is the first or not. From my point of view, the important information is that in 2011 there was a synthetic windpipe transplant done in a hospital and it's one of the first such transplants. That's the information I would be interested to add into the Wikipedia article. If it's truly the first or not, it doesn't really matter. We'll find that out later from review articles. Until we find that (it can take decades), we better have that information instead of not having it.

I am in no way advocating for use of popular press instead of review articles. By the contrary. But for me it's easy to find such data in popular press and it's very difficult to find it in review articles. I don't have the skills and time to do find such articles. I haven't even added such information in English Wikipedia articles. At least not yet. I advocate that we can use (or should allow the use of) popular press until we find the same (or updated) information in review articles - in such cases of hospital firsts, and not for information about laboratory experiments. And when the information from popular press proves right, instead of replacing the BBC reference with the new one, we should just add the new reference in front of the BBC reference. I noticed that a lot of references from Mayo Clinic are dead links already (e.g. Thermoregulation - [6] and Calciphylaxis - [7]). Therefore it would be nice to have another backup reference from popular press that tells the same story for every Mayo Clinic link. The same might apply to many other academic articles and to many other review articles. I don't care about recentism, I do not care if such information is about what happened one minute ago or 500 years ago. I just want to know things and to make knowledge available, whenever possible. If the Wikipedia rules allows to make a particular information available into an article or at least into a talk page, I will try to add it into Wikipedia.

If someone deletes such information (in a Wikipedia article) because is referenced from popular press and then someone adds the same information after 20 years, when such information is available/found in review articles, then I find the justification to delete it a worse justification than the one you pointed in 1) (easier to find with Google). In those 20 years we better have some information about the topic instead of none.

Speaking about Recentism, the Trachea article has such a reference from 6 years ago - [8]. Is that recent? I'm not arguing here, I just ask because I really don't know. --  Ark25  (talk) 11:43, 22 June 2014 (UTC)

The Sweden synthetic windpipe transplant is an experiment too, I agree with you. But it's also a public event (takes place in a hospital), unlike laboratory experiments. And is it an important milestone (or at least an important event), even if it fails in the end. Such an event deserves it's place the history of synthetic windpipe transplants, even if it's not the first but it's the third of fifth such event, do you agree with that? --  Ark25  (talk) 14:43, 22 June 2014 (UTC)

Here are my thoughts:

• Let's have historic information, properly placed under ==History==: If a (decent) newspaper says that a historic surgery took place at the Foo Hospital in 2011, then let's have that information (in a suitable article).
• Let's upgrade your skills: It's pretty easy to search http://www.ncbi.nlm.nih.gov/pubmed/ You can go there and search for synthetic trachea. Then, in the left margin, click on "Reviews" under "Article types", and if you want, "Free full text" under "Text availability". That should get you three papers: PMID 24059453, PMID 23184357, and one from 1984(!) that you'll want to ignore.

The combination of these two approaches will often get us decent historical information. (You can also look into the "lay" parameters for {{cite journal}}, which allow you to cite both an academic paper and a related news story. I'd be happy to see someone working on this subject, because the history of surgery is one of the areas that is a bit neglected on Wikipedia. WhatamIdoing (talk) 04:49, 23 June 2014 (UTC)

WAID so many times I find you wondrous, including this one. Jytdog (talk) 05:24, 23 June 2014 (UTC)

Thanks for the answer. My English is not perfect. I changed now the I advocate that we should use (..) popular press into I advocate that we can use (..) popular press because that's what I meant to say there. I am not sure that I can have the time to add such historic facts about surgery in the Wikipedia articles but maybe I will have the time to notice them on talk pages, to serve as a starting point for those who might have the time to add them later in the article, including for me when I will have more time. WAID means WhatamIdoing, for those who are wondering what that could possibly mean. --  Ark25  (talk) 08:51, 23 June 2014 (UTC)

WhatamIdoing - I did the search that you suggested me to do, and indeed I found the three results you said. And then I searched for "face transplant" and I got 74 results. Most of the results contain things like "Liver transplantation", "kidney transplant" and "renal transplant". Is it possible to get better search results? I would like to see if I could find there references for the 6 face transplants presented in Face transplant#In Turkey. --  Ark25  (talk) 13:34, 26 June 2014 (UTC)

Well, some of them are just going to give you a long list no matter what you do, but one thing to try is this: click on one that seems plausible. Underneath the text of the abstract, there should be a section called "Publication Types, MeSH terms". Click on that, and you'll see a list (sometimes incomplete, so you might have to check a couple) of "subjects" like "Face transplantation". (This is also where you figure out if a paper is a review.) Clicking on your favorite of those subjects will search for only articles with that subject tag (e.g., http://www.ncbi.nlm.nih.gov/pubmed?term=%22Facial+Transplantation%22%5BMeSH+Terms%5D for facial transplantation), which will help eliminate irrelevant papers. In the search box for that search, you can also add things, like the word "Turkey", and you can use the filters on the left sidebar to choose dates and reviews, etc. Using this system, I found that PMID 22894997 mentions face transplants in Turkey (but I don't have a copy of the full paper). WhatamIdoing (talk) 16:30, 26 June 2014 (UTC)

src: upload.wikimedia.org

Science by press release in Neonicotinoid article

See here. I posted there, instead of here, because this guideline is not written to clearly read on veterinary medicine. But the issues are similar. Jytdog (talk) 12:24, 1 July 2014 (UTC)

src: upload.wikimedia.org

Surse de încredere (medicin?)

Hello. I'm pleased to announce you that a carefully translated, adapted and slightly completed version of the WP:MEDRS was adopted as ro.wikipedia guideline. Our Wikipedia may be small, but is now the 4th to have such a page. However, there are some notable differences as well. I considered more appropriate dividing the section Choosing sources into only 2 simple subsections: medical and non-medical sources. I believe that in an attempt to offer as much as possible, MEDRS stepped over the bounds of "general guideline" and penetrated a bit too much into the research field. It's a good thing, no doubt, but I would rather see such information in an additional page like an essay or supplement. --Winter^eu (talk) 00:49, 9 July 2014 (UTC)

Great to hear that you have found it useful. Doc James (talk · contribs · email) (if I write on your page reply on mine) 15:55, 9 July 2014 (UTC)

src: upload.wikimedia.org

News article

There are questions on whether or not this article [9] can be used to support "However, another article in Nature, stated that acupuncture activates the brain, and does more than placebos" here [10]. Thoughts? Doc James (talk · contribs · email) (if I write on your page reply on mine) 04:01, 25 June 2014 (UTC)

No placebo iz for acupuncture, because you feel treatment: Interviewers can be blind, not subjects. You can do a case-cohort study in this, with matched controls that are about the same before treatment. To do a good job of it, the interview would need to be conducted in writing, because many tonal ways are to let on whether you are case (treated) or cohort (placebo). Hmm. Wait a minute. I wuz going to say that machines could do the interview in an objective manner, and the key words are 'activate brain', so perhaps machines already are doing the interview in the form of Magnetic Resonance Imaging, Positron Emission Tomography, or Electroencephalography. I would say that I am beaten after actually reading the article, except for this question: Doesn't real pain always cause endorphin release from pain moderation centers in the brain? The problem with the article is that it confuses placebo with accupressure: No placebo is for accupuncture. 75.152.119.10 (talk) 10:33, 13 July 2014 (UTC)

Hey Doc James. Nope, that souce should not be used. That is a news piece covering publication of a primary source, and MEDRS specifically speaks to why we shouldn't use those (as you know). On contentious articles we should strive to use the best sources we can find; reaching for primary sources is a red flag for POV pushing (generally, but especially on contentious articles where everybody should be reaching for sources that people with differing viewpoints will find valid). Back in the archives of the acupuncture Talk page, somebody proposed this as a source, which I thought was great. It is a meta-analysis of fMRI studies of people undergoing acupuncture published in 2012. (I was working on that article until QuackGuru drove me crazy and therefore away, btw.) Jytdog (talk) 22:54, 28 June 2014 (UTC)

I'm not convinced that Nature.com is what we meant by "popular press", although I wouldn't describe it as "another article in Nature", since that will imply a peer-reviewed article to some readers. WhatamIdoing (talk) 19:45, 29 June 2014 (UTC)

I see little doubt about whether acupuncture or accupressure work. There's also a plausible mechanism, so this issue iz not about POV-pushing. There's probably a machine to do accupressure on the head with a technician instead of a neuro-anatomist. Transcutaneous_electrical_nerve_stimulation machines use the same nerve endings, and last time I checked, they require anatomists to operate. 75.152.119.10 (talk) 10:33, 13 July 2014 (UTC)

src: upload.wikimedia.org

User essay - Why MEDRS?

So I created an essay in my user space about why we need MEDRS, stated in plain language as much as I could. It is waay too long, but it is here: Why MEDRS? Feel free to mess with it. Or ignore it. Jytdog (talk) 20:26, 21 June 2014 (UTC)

Hi Jytdog: Thank you for this essay. There is some good stuff in it. I agree that it is way too long. Could you edit it down please? I suggest in particular that you include less on examples of "biology is difficult" and add a succinct paragraph or two on the implications of this observation to MEDRS. CBHA (talk) 20:07, 25 June 2014 (UTC)

• Support I think this essay is an excellent idea, and something we've needed for a while to direct new editors to. The only objection I have is that it focuses largely on biology and not medicine, and that isn't really something that MEDRS covers. Non-medical biology falls under the essay WP:SCIRS, but that is also a different discussion entirely. -- CFCF (talk · contribs · email) 21:39, 29 June 2014 (UTC)

Thanks for the kind words CFCF. The reason I focused it on biology, is that medicine is to biology as tech is to physics. Medicine is biological technology. It is because our knowledge of biological science is so fragmentary and slim, that medicine is so often such weak technology. (unlike physics/material science, which is very advanced science, which has in turn allowed rapid advances in technology (eg. Moore's law) and almost magical devices like smart phones to come into existence) This is the key thing that so many people cannot wrap their head around. How much doctors don't know, and how little they can actually do in many cases. (the joke about neurologists: Two doctors are up in a balloon and are lost and call down to someone on the ground, "Where are we?" and the guy hollers back "In a ballon about fifty feet off the ground!" One doctor turns to the other and says "Neurologist - told us exactly where we are and didn't help a bit.") I guess I need to make that more clear and maybe focus more on the medical uncertainty itself. On the other hand, I want to make it super clear why bringing a primary source discussing mouse or in vitro work to support health-related content in an encyclopedia is just ludicrous.... Jytdog (talk) 21:59, 29 June 2014 (UTC)

I agree that those are valid points, and I only meant to point out how it's odd we have MEDRS for medical content, but no real guideline for other (non biomedical) biology content, and that your essay makes a strong point for having an equivalent WP:SCIRS guideline (currently only an essay), or maybe even making a new WP:BIORS/alternatively merging any such guidelines with MEDRS.

P.S. Liked your joke :). -- CFCF (talk · contribs · email) 22:16, 29 June 2014 (UTC)

• Support. Agree that an essay like this could serve a real purpose within the community. It would be good to get it nicely focused so that it can address in a concise and easily comprehensible the sorts of questions that Wikipedia's broad community of editors commonly entertain. I too am not sure whether "biology is difficult" is the best starting point (isn't biology broadly covered by the SCIRS essay anyway?). Perhaps a key issue here is the fallacy that findings at a microscopic level (or in animals) can be directly extrapolated to human health? 86.128.169.211 (talk) 11:35, 30 June 2014 (UTC)

• Abstain. The problem with 'this' article is not whether we need it or not. The problem is not that we are directly extrapolating anything to human health. We never did. We never will. Sometimes, we let readers decide relevance. The problem iz that it does not call for any thinking. Delete primary sources. Delete studies in glass. Delete double-blind, placebo-controlled study if it found significance where a meta-analysis that included industry funded sources did not. Assume that twenty million abstracts in PubMed are the whole truth. 75.152.119.10 (talk) 10:17, 13 July 2014 (UTC)

• Support useful essay, although could be tightened up. Zad68 23:14, 13 July 2014 (UTC)

src: upload.wikimedia.org

Differences between WP:MEDRS and WP:SCIRS: is there are line drawn somewhere between the two?

I ask this because I deal with science related articles, and a lot of my background comes from toxicology, veterinary medicine, etc. In these cases we're not dealing with human health, but rather animal health. It would seem pretty clear that WP:MEDRS applies in this situation because we are dealing with medical research, but this does seem to irk some editors when it's cited because it's not human medical research. In animal research, you can move away from more medical topics and get into things like ecology and WP:SCIRS is a better fit, but it's also not a guideline like MEDRS is. Is there a distinct scope to MEDRS where it's meant only for human health? If not, is there a cutoff somewhere in the general biological sciences? I'm not seeing that after reading over the page since the spirit of the page seems to be aimed at any kind of medical research or health claims. With that in mind, should we clarify in the page text what MEDRS actually applies to? There's definitely a blurry line between medical and natural sciences in some cases, so I'm not sure how a good statement of scope could be easily done.

In all honesty though, it seems that MEDRS and SCIRS hold very similar standards for saying what is a reliable scientific source. If I'm dealing with plant health, the same aspects of what's a reliable source in animal medicine still apply (e.g. primary vs. secondary literature). However, plant health is even further removed from human health, so someone could claim MEDRS isn't applicable, and I'm not aware of any guidelines that are as informative as MEDRS in that case. For editors that get really hung up on what medicine actually pertains to, should we be adding some clarification in the guideline by having a nonrestrictive list of example fields (toxicology, veterinary medicine, etc.)? Maybe there's a better way to address editor questions of whether MEDRS really applies an article, but I thought I'd float the question out there since I'm sure other folks have run into similar issues. Maybe there's some convention other folks have followed for this kind of gray zone that might be worth spelling out somewhere?

I know some editors are uncomfortable with the MEDRS guideline (maybe because it's a new concept to some), so there can be cases where someone might want to avoid the guideline by saying the topic technically isn't within the scope. In a case like that, I'm wondering how do we concisely point out what a reliable source is for scientific content in general without people getting hung up on the name of medicine or natural sciences? Is there something that can be done in this guideline? Maybe work on getting SCIRS to guideline status (might be tough looking at previous discussions), or maybe something at WP:RS instead? Maybe it's entirely splitting hairs and MEDRS just gets informally applied to all "hard sciences" (not going to touch where that cutoff is) in practice, but since some of the questions apply to this page, and others are interrelated with SCIRS and RS, I figured I'd start by asking here and seeing if certain questions might be more appropriately addressed here or at the other pages. Kingofaces43 (talk) 16:51, 27 June 2014 (UTC)

I don't think you're alone in having a hard time separating when these different guidelines are applicable. There is currently ongoing discussion over at WT:MED#Issues with alt med articles & Wikipedia talk:Surrogate endpoint, although they might not primarily focus on this distinction it has been brought up. The difference as I see it is that WP:MEDRS is applied far more vigorously due in part because of the strong commitment from the WP:MED community. I think there should be a clear distinction between MEDRS and SCIRS in the lede of both guidelines and think this would be a good time to bring up the discussion, when there is already so much discussion concerning MEDRS.

I think we need to define what a medical article is. When you think about it, it isn't so very straight forward. Anatomy and physiology are in part arguably closer to biology than medicine, even if medicine builds on them. -- CFCF (talk · contribs · email) 12:10, 29 June 2014 (UTC)

It is not a question of a "medical article" but rather health/biomedical content in any article. The lead of MEDRS was written carefully to cover a wide scope of content. And yes MEDRS evolved because there is so much bad content out there, and so many editors who come to WP wanting to push that content into articles, and so many people look to WP for health information, that the community needed a way to manage the crush and live up to its mission of providing reliable information to the public. On the scope issue, see for example this response from Colin who I understand was part of the group who originally drafted MEDRS in response to that need, and especially this comment by him too, and this comment by WhatamIdoing who I understand was also part of that team. These issues come up all the time and the archives are rich with discussion of them. Colin doesn't show up here much anymore much but WAID does, and what each of them have to say generally provides really great insight into the experience and reasoning by which the community got here. And look at this article from mercola.com for example -- do you see how they use primary sources and claims about biochemistry to make really outrageous health claims? That is what we have to contend with in the health arena. Jytdog (talk) 14:48, 29 June 2014 (UTC)

The issue however is not how sources are (ab)used outside of WP but how they are used/should be used within.--Kmhkmh (talk) 15:00, 29 June 2014 (UTC)

the point of my bringing the mercola article was twofold: first, to show the kind of thing that editors bring here and want to cite as a reliable source and that we need to keep out... and secondly, to show the way that what might appear to be basic science research (biochemistry etc) can be abused to generate health claims and thus why the lead of MEDRS is purposefully written broadly. Jytdog (talk) 15:14, 29 June 2014 (UTC)

Yes but using sources in proper or improper fashion is somewhat of separate issue. Because the problem is not the primary but its improper use. You can abuse secondary sources as well, though I agree it harder to do that. My point however is that should focus more on the appropriate use of reliable sources rather than handling it indirectly by categorically excluding primary sources.--Kmhkmh (talk) 18:27, 29 June 2014 (UTC)

It might be wortwhile to note that WP:MEDRS is guideline whereas WP:SCIRS is merely an essay.--Kmhkmh (talk) 14:56, 29 June 2014 (UTC)

Kingofaces43 there are processes for promoting essays to guidelines, and guidelines to policy. I am not familiar with them but WAID or other much more experienced editors may be. As to where the line is, we have both seen people say things like "the health of bees is not subject to MEDRS" which I really struggle with, since the difficulties are very similar. The topic of whether MEDRS should apply to veterinary medicine has been brought up and never resolved. I would favor MEDRS making reference to vet med. I will draft a proposal for everybody to consider... let's try again! Jytdog (talk) 17:23, 29 June 2014 (UTC)

The spirit of MEDRS is that we do not give false information about medicine to people who may rely on it. Of course we are not supposed to do that for any subject. But unless someone can show that a specific statement is medical information, MEDRS guidelines do not apply. Content policy, if followed, should be sufficient. Sources policy dictates that articles are primarily based on secondary sources and that the best sources are used. NPOV requires that articles provide proper weight to different aspects of subjects and opinions. So if a single study shows that a certain diet may increase your cat's life by ten years, it should not be added unless there is strong support for the finding among veterinarians. TFD (talk) 18:59, 29 June 2014 (UTC)

Colin started MEDRS; I joined in later.

I wonder if it would be possible to merge MEDRS and SCIRS? The structure is very similar. There's also WP:SCICITE, whose value I question (but perhaps I just don't appreciate it enough).

On the original question, I'd use SCIRS for ecology statements like "a spill of scarychemicol was blamed for the drop in amphibian reproductive success at Lake Wobegon" or "bumblebees communicate by wiggling their abdomens". I'd use MEDRS for anything that a typical reader might believe was related to human health. WhatamIdoing (talk) 20:11, 29 June 2014 (UTC)

and i should say that in my experience with them, Colin has generally been more hardcore in applying MEDRS, while WAID has generally been more flexible, especially with regard to use of primary sources. (not sure if you would agree, WAID, but that is my perception). But WAID, if we merged SCIRS and MEDRS, where would that leave animal health matters as per your two examples? How would we handle that? Jytdog (talk) 20:59, 29 June 2014 (UTC)

I think merging the guidelines/essay is an excellent idea. I've come across the argument that something is biology/physiology/neuroscience/research, and not really medicine or clinically related, and therefor MEDRS shouldn't apply, one to many time. As said, much is similar, but it won't be a walk in the park merging them either, but I fully support it and would do my best to help out. -- CFCF (talk · contribs · email) 21:14, 29 June 2014 (UTC)

The problem however is that depending on the context that is actually a valid argument and therefore probably a reason for not merging those 2.--Kmhkmh (talk) 21:21, 29 June 2014 (UTC)

Merging them might not be good idea, at least if it supposed to cover all areas of science and if it supposed the medical aspects only I see no reason for merging, there we can simply use MEDRS as it is.--Kmhkmh (talk) 21:18, 29 June 2014 (UTC)

WhatamIdoing, here's an interesting intersection from my neck of the woods. Take toxicology as an example. What makes a toxicological source reliable is based upon things we're familiar with in health in general (e.g. dose-response, biologically relevant doses, efficacy of a drug/pesticide, etc.) and reliability of experimental results are determined through proper experimental design, replication, inclusion in secondary sources, etc. Basically what we find reliable in toxicology for sources isn't really any different than what we have in MEDRS right now (likely because toxicology plays a large role in medical research, human or otherwise). Ultimately toxicology will always come back in some form towards assessing the health of an organism. Where exactly would this topic fall then? It's definitely used in ecology in some areas, but we're still talking about health and medicine in various forms in the end. MEDRS currently doesn't specify it's intended just for human health, but I don't see any reason why it should only apply to human health either. The qualities that make a human medical source reliable typically are going to be the same ones we use in veterinary medicine as well. Veterinary medicine can be broad too as it covers not only traditional farm livestock, but insects (bees especially), wildlife veterinarians, etc. I'd even argue that many other branches in the life sciences use those same qualities for reliable sources too, so I'd agree with folks who think SCIRS and MEDRS should be merged as there's a lot of redundancy in the two, but I'd rather get the seemingly simpler question of where veterinary medicine fits in to MEDRS before worrying about even tougher things to work out. Kingofaces43 (talk) 06:27, 3 July 2014 (UTC)

Forgive me if this seems pedantic, but what makes a source reliable in toxicology is whether it's published by an outfit with a reputation for fact-checking (e.g., anything with regular peer-review) and the other qualities listed at WP:NOTGOODSOURCE. Things like whether it used biologically relevant doses is (a) something that makes it good or not, and (b) not something that editors should be determining on their own (that job belongs to the secondary sources that we hope editors will cite). WhatamIdoing (talk) 21:00, 3 July 2014 (UTC)

Oh no, I definitely agree with what you said. I didn't intend to lump reliable vs. good sources together both as something editors should be addressing, but rather that what makes a source good in the scientific realm are qualities like good experimental design, biologically relevant treatments, etc., that means the primary source gets cited (ideally) by secondary sources, and then we at Wikipedia consider the source reliable. Basically I was getting at those inner workings that happen in the science realm before we start considering information's role in Wikipedia. The two are intertwined, so sorry if what I was getting at wasn't clear. What makes a source good for scientists, or rather specifically what issues come up from focusing too much on a primary journal article are why we have guidelines like MEDRS for specifying what is reliable at Wikipedia. MEDRS does a good job of giving a brief background on what makes a source good or bad for the medical community and how issues there with a source translate into reliability issues here at Wikipedia.

With all that in mind, what I'm ultimately asking is whether, as an example within toxicology, a topic like cattle toxicology should be be covered by MEDRS when human toxicology obviously is? Both are medical fields. The inner workings of the scientific research and the validity of findings that eventually can lead into acceptance or rejection by the scientific community, and what eventually translates into whether a source is reliable on Wikipedia are not substantively different as far as I've seen. In the end, I'll be a happy camper if two questions get discussed here and hopefully lead into whether MEDRS is better intended to have a limited scope within certain branches of medicine, or if it is actually set up to be applied more widely within science research than some people think:

1. Does MEDRS apply to non-human branches of medicine?

2. If not, why?

Thanks, Kingofaces43 (talk) 21:57, 3 July 2014 (UTC)

I don't think that the community has an answer to your question.

One possible concern is whether vet med goes in for review articles as much as human med does. We don't really want to restrict the number of sources down to the point that you can't write an article about (for example) cattle diseases in the first place. WhatamIdoing (talk) 00:59, 9 July 2014 (UTC)

Agree: It is important to use the "best" sources, but sometimes those sources are not the best possible. Don't want to make the perfect into the enemy of the good. An analogous situation to this is some edit warring I have seen in the organic food articles, where MEDRS was used not only for medical claims, bur for other claims regarding, for example, water pollution, which clearly was beyond the scope of MEDRS. I have also seen the policy similarly abused to include studies paid for by Monsanto, but exclude other sources clearly acceptable per WP:RS and WP:SCIRS. Another problem I have seen is the pseudoscience wars where instead of simply putting due weight on fringe theories, MEDRS was used to exclude any discussion of different views and as a result created a wholly biased article stating that the entire concept is, essentially, worthless. This would not be a worry for, say, tinfoil helmets, but it is when you run across people who apply MEDRS to new things that haven't had a lot of peer-reviewed studies done yet, such as veterinary chiropractic and such. Montanabw^(talk) 20:00, 14 July 2014 (UTC)

Is including content because of a lack of reviews taking priority over WP:FIVEPILLARS though, namely the spirit of WP:IINFO and WP:NOTE for encyclopedic content? One of the core foundations of Wikipedia is that we are using reliable sources, and we don't include information simply because it is there. I'm a little iffy on changing standards across topics because one isn't written about as much in reviews (seems like a notability issue more than anything). I haven't particularly seen issues with with the lack of available secondary sources in veterinary topics (we typically have plenty of review papers in general), so maybe we're bumping up against notability issues more than reliability in this context? Kingofaces43 (talk) 15:24, 15 July 2014 (UTC)

TFD, I definitely agree that the points on content policy you mentioned should be sufficient (although it is difficult to get across to some editors). You mentioned showing that we need to show that a statement is medical information to apply MEDRS, but what exactly is medical information? Medicine definitely isn't limited to human health in the real world, and MEDRS isn't specifically singling out human health the way it is currently written since it simply addresses medicine and not human medicine specifically (although it is written with some examples of it, but much of the guideline isn't that specific). Now I will say that many people do mistakenly assume human health is being talked about when the word medicine or health is used, which is probably why there has been a little contention on this topic. Because of that though, we can have people rejecting MEDRS on non-human health related topics even though the reliability of sources are essentially determined the same way as human health sources in the real world. It seems like that unintentional bias of associating health source reliability just with human topics makes discussions much more difficult in other health related content areas. That's where my concern lies. It's definitely doable to go through normal content policies and guidelines, but it does seem like it can lower the bar a bit and muddle the discussion when MEDRS is ignored for some health related content.Kingofaces43 (talk) 06:27, 3 July 2014 (UTC)

SCIRS looks a great deal like MEDRS for the very good reason that I structured the former to follow the latter. Last time I tried to put in the legwork to get the essay up to guideline status, drama ensued. Anyone is welcome to try, as many of the same concerns apply there as here. - 2/0 (cont.) 15:42, 14 July 2014 (UTC)

In discussion below about alt-med topics I proposed "When it comes to reviews of medical research, use the guidelines contained in WP:SCIRS instead.". This would allow a more permissive policy that would allow minority views in medical research to be presented without the almost no possibility of a incorrect information being used as medical advice. - Technophant (talk)

src: upload.wikimedia.org

Prediabetes

The contents of the Prediabetes article seems to be outdated, see here. Count Iblis (talk) 17:41, 16 July 2014 (UTC)

src: upload.wikimedia.org

Issues with alt med not being able to maintain NPOV

Surely there has to be some compromise between a strict legalistic view of MEDRS and the mandate to have a neutral point of view. We could show that while the medical consensus is this, other opinions do exist. This in no way would affect the accuracy of the medical information in the article, indeed it would enhance it. An example is being played out in Acupuncture with a growing body of evidence on the mechanisms of how it works being whitewashed because an outdated reviews says there are none. One solution is to hold the content dealing with research and possible mechanisms of action to the standard of WP:SCIRS while the content that could be used as medical advice retaining the MEDRS standard. - Technophant (talk)

I don't believe that MEDRS can be re-written to eliminate POV pushing. WhatamIdoing (talk) 04:03, 20 July 2014 (UTC)

There's not really a discrepancy between MEDRS and NPOV. NPOV stipulates that we not give undue weight. Using unreliable sources for content is a kind of POV issue, namely when using primary studies because they typically don't describe the scientific consensus. Basically if the consensus is indeed changing, you should be able to source that in appropriate secondary sources. Without that secondary source, weight cannot really be assigned, which violates NPOV. Simply having an opinion exist isn't grounds for including it on Wikipedia. If that weren't the case, we wouldn't have WP:Fringe specifying that certain opinions be excluded. Kingofaces43 (talk) 04:39, 20 July 2014 (UTC)

Some examples of this kind of white-washing is this and this. I found a reliable secondary source that does acknowledge that opinions are changing. See this article from the WSJ. It says "Scientists are also finding parallels between the ancient concepts and modern anatomy. Many of the 365 acupuncture points correspond to nerve bundles or muscle trigger points. Several meridians track major arteries and nerves." Critics however have rejected using this source because of MEDRS.

I don't have evidence that consensus is changing, however there's credible research by reputable scientists published in respectable journals such as the Journal of Pain. When these ideas are picked up on by a major news organization, researched and fact-checked, the standard of reliability should be met, however the popular interpretation of MEDRS doesn't allow this. Something is wrong here. - Technophant (talk) 04:47, 20 July 2014 (UTC)

The main issue appears to be where you are using the journal articles as primary sources you are citing, and the WSJ isn't a reliable source for synthesizing primary literature. Technically, the news organization is picking up on the primary study and is not what is considered a scholarly secondary source. When it comes to scientific content, that doesn't particularly give weight to such studies. It's other scientists citing the studies and indicating validity in their own papers, especially reviews, that are needed. There isn't anything wrong with that, that's just the nature of how scientific publishing is done. The main issue is that primary studies require interpretation, and news organizations are not particularly adept at doing that, and we as editors are not allowed to do that even if we are a scientist that publishes in journals as well. For someone not too familiar with scientific publishing who has already learned the general guidelines of Wikipedia for reliable sources, this can definitely be confusing, and you're not the first to have difficulty with this topic. Hopefully that clears things up for you a little bit. Kingofaces43 (talk) 05:13, 20 July 2014 (UTC)

I propose this change to Acupuncture#Scientific view on TCM theory: "A 2008 review of research did not find a consistent anatomical structure or function for either acupuncture points or meridians. However, a 2010 article in the Wall Street Journal stated that "Many of the 365 acupuncture points correspond to nerve bundles or muscle trigger points. Several meridians track major arteries and nerves." That does not put undue weight on recent research and allows to reader to look into this and make their own decisions. - Technophant (talk) 06:02, 20 July 2014 (UTC)

That's a horrible proposal. You're proposing to allow a nonscientific source like a newspaper article trump a scientific review of research. The very sentence construction is designed by you to lead the reader to believe that because a newer source says something different than an earlier source, it's more likely to be true, and you set up and create the reader's expectation by using "however".

That's a textbook example of POV editing and synthesis. This proposal is very telling about your mission here at Wikipedia, and it's not a good one. No wonder you're running into trouble everywhere you spam your complaints. You are becoming an enormous time sink. -- Brangifer (talk) 07:12, 20 July 2014 (UTC)

It does put undue weight on a single study. Reviews and meta-analyses are high quality. A newspaper assessment of the results of a single study carries virtually no weight in comparison. The statement, "I don't have evidence that consensus is changing" pretty much sums it up. Have basic anatomy texts begun including meridians? Are meridians part of serious academic discussions of physiology? The mainstream scientific consensus and current medical knowledge are what WP reflects, not a single study that got some press. When the consensus changes we can herald Dorsher as one of those who paved the way, until then WP reflects mainstream predominantly and FRINGE as due. I have to agree with BR your editing is becoming tenditious with more than a whiff of POV PUSHING and a growing stink of IDHT. - - MrBill3 (talk) 07:36, 20 July 2014 (UTC)

Calling someone names and repeating your arguments over and over again isnt going to make your point any stronger. Yes, meridians have been accepeted in some very high-quality reviews, including that of the Mayo Clinic Proceedings and elsewhere. In fact, consensus has changed everwhere you look, but I'm skeptical as to whether I should waste my time discussing the same issues with the same editors over and over again. -A1candidate (talk) 09:22, 20 July 2014 (UTC)

What has been repeated? It seems to me policy based arguments have been presented by a number of editors with clear explanations of how they relate to WP policy. Rather than a link to the WP article for a journal perhaps you could provide the pmid's of these high-quality reviews. If consensus has changed everywhere you look, perhaps you could provide the isbn and page number of the major anatomy and physiology texts that reflect this supposedly widespread change of consensus. If it has changed everywhere I would expect to see statements of major medical organizations and meta-analyses (perhaps some comparisons of various ways of delineating meridians and the validity and repeatability of these), probably editorials in the major journals discussing this change of consensus. Sources please? - - MrBill3 (talk) 10:02, 20 July 2014 (UTC)

@user:A1candidate Agreed, however I do think this is an issue worth discussing. - Technophant (talk)

• Proposal to change policy to say "When it comes to reviews of medical research, use the guidelines contained in WP:SCIRS instead." - Technophant (talk) 06:08, 20 July 2014 (UTC)

Dead in the water. The point of discussing research as you want to, is that it goes toward health claims; if it didn't, you wouldn't be pushing to include it. When Wikipedia discusses health issues, we have a responsibility to provide the public solid information based on the scientific consensus, as well as that can be established. WP arrived at this place through a lot of hard work and difficult discussions. What you propose opens the door not only to issues you care about, but buckets of woo -- things like HIV/AIDS denialism, Vitamin C megadosage, Magnet therapy, etc etc. As per Thomas More in Robert Bolt's A Man for All Seasons: "do you really think you could stand upright in the winds that would blow then?" Jytdog (talk) 06:43, 20 July 2014 (UTC)

I can't stand upright in the winds that blow now. It's very discouraging to attempt to add information to article in a way that would be perfectly acceptable in any other scientific article and have it reverted and insulted by multiple editors claiming to be using the "correct" interpretation of medrs. Until the policy is rewritten Wikipedia will continue to have a conservative bias- - Technophant (talk) 06:54, 20 July 2014 (UTC)

But it should have a conservative (small c) bias: the task of Wikipedia is to convey established knowledge. This is particularly important in medical matters, where we know that desperately ill people clutch at straws and can even be defrauded by charlatans. Peter coxhead (talk) 08:46, 20 July 2014 (UTC)

We're here to summarize scientific consensus. We're not here to fight assumed "fraud". -A1candidate (talk) 09:33, 20 July 2014 (UTC)

See WP:FRINGE, "A Wikipedia article should not make a fringe theory appear more notable or more widely accepted than it is." and "Proponents of fringe theories have in the past used Wikipedia as a forum for promoting their ideas. Existing policies discourage this type of behavior" and "Articles which cover controversial, disputed, or discounted ideas in detail should document (with reliable sources) the current level of their acceptance among the relevant academic community." and "Ideas that are of borderline or minimal notability may be mentioned in Wikipedia, but should not be given undue weight." and well perhaps read the guideline. - - MrBill3 (talk) 10:11, 20 July 2014 (UTC)

Scientific consensus isn't a fringe theory. -A1candidate (talk) 10:57, 20 July 2014 (UTC)

But we're discussing a fringe theory which is definitely about as far from the scientific consensus as the east is from the west. -- Brangifer (talk) 16:08, 20 July 2014 (UTC)

@A1candidate: We seem to be misunderstanding one another. I entirely agree that the task to summarize scientific consensus. I was talking about the motivation for WP:MEDRS being different from WP:SCIRS. Summarizing scientific consensus as accurately as possible means using sources that are reliable in relation to the branch of science involved. Scientific research which requires statistical analysis of sufficiently large samples is different from scientific research which can be based on small numbers of experiments or observations.
It's also perfectly reasonable to require the quality of the evidence to be related to the consequences of the outcome. In practical applications of science, such as medicine or engineering, there's a precautionary principle (mandated safety margins added to scientifically determined failure stresses in construction, for example).
However, we should not suppress information. I have been concerned in the past that WP:MEDRS can be used to prevent the reporting of historical or current belief in, or use of, non-scientific medical treatments, and this is wrong. We must never appear either to condemn nor to endorse any potential treatment; merely report what reliable sources say about it. It's a tricky balance to get right, and of course we don't always succeed. Peter coxhead (talk) 11:09, 20 July 2014 (UTC)

One common tactic of POV pushers is to delete ALL primary sources under the guise of Medrs. This isn't in spirit of the guideline or the letter of the law. - - Technophant (talk) 11:40, 20 July 2014 (UTC)

Yes deleting all primary sources is a great way to improve the overall quality of sources used. Secondary sources do usually provide a much better overview of major positions. Doc James (talk · contribs · email) (if I write on your page reply on mine) 12:44, 20 July 2014 (UTC)

Technophant, when you wrote that you can't stand upright now, you ignored what I wrote. Surely there are approaches to health you see as snake oil. Do you want those approaches granted credence on WP? That is a real question! Please think about the big picture, which MEDRS must address. Moving on, in my view, Technophant, even just today you cited Gandhi's quote encouraging those fighting for change. Gandhi was out to right great wrongs in the real world. Wikipedia is not a vehicle for advocacy. We express the scientific/medical consensus as stands to today. (The scientific/medical consensus does not change rapidly - WP does have a small "c" conservative bias on health matters!) Acu may have solid scientific foundation one day, but that day has not arrived. And until it does, acu as a field remains pseudoscience and FRINGE. (some applications appear to have been empirically validated or very close to it; that is different from the approach itself having a widely accepted scientific basis.) WAID's initial comment was dead on - changing MEDRS is not a solution for what are essentially NPOV problems that arise on all sides of alt health issues. Jytdog (talk) 13:58, 20 July 2014 (UTC)

+----------------------------------------------------------------------------------------------------+@user:Jytdog If there is peer reviewed research that snake oil works let it so be noted (however that's not going to happen). I'm a man of science and have faith in the scientific process. Many young researchers read Wikipedia and get ideas on what they want to do doing their scientific careers. I've Wikipedia leaves out the known but unconfirmed and only reflects mainstream then it's article on medicine will remain short and incomplete like they are now. - Technophant (talk) 14:53, 20 July 2014 (UTC)

Please see WP:MEDMOS and its description of how to handle ongoing research. It is fine to include that kind of information in that way. On a deeper level... basic research in the biomedical space is published by scientists for scientists, as they push the boundaries of their field and talk to one another. We live in a day when we can watch that conversation unfolding. But we have to respect it for what it is -- exploration. Many published studies end up being unreplicable or draw conclusions that turn out to be dead ends. Those publications are rarely withdrawn; they are just ignored by the field as it moves ever onward. Review articles tell us where the field stands - what ideas or approaches have turned out to be productive, and which were barking up the wrong tree. It is really an abuse of WP and of the scientific process to grab this or that primary source and decide - on your own authority - that this one is durable or right or important. Rely instead on reviews in which experts in the field judge where the field stands. That is what MEDRS is all about. Please reach for WP's highest standards, not its lowest ones, to help achieve our mission of providing reliable, NPOV information to the public. Jytdog (talk) 15:35, 20 July 2014 (UTC)

(edit conflict) :This reminds me of a story I heard about a conference on vitamin supplements. Presentation after presentation was about how supplementing this or that vitamin was worthless or harmful. At the end, the speaker said, "Well, we've heard a lot of good research this week, and none of it showed any value in taking vitamins. But let's have a quick show of hands: How many of you take a vitamin each day?" Almost every hand in the room went up. So what actually was "the scientific consensus": what these experts said in public about vitamins, or what these experts personally did about vitamins? I can tell you this with certainty: the editors who believe widespread use of any product by millions (or, in the case of TCM, more than a billion) of people means no acceptance for this FRINGE practice will tell you that the scientific consensus staunchly opposes vitamin supplementation, even though nearly all of those same vitamin experts are taking them. WhatamIdoing (talk) 16:05, 20 July 2014 (UTC)

@WhatamIdoing: The scientific consensus doesn't, so far as I can tell, "staunchly oppose" vitamin supplementation, it merely says that there's no evidence for its effectiveness when properly tested on large samples, which is quite different. Peter coxhead (talk) 16:32, 20 July 2014 (UTC)

WhatamIdoing, I'm not certain I understand your point (I suspect you switched a couple terms, making it illogical), but I think I know what you're attempting to say. Let my give it a try:

• If millions of believers have used a method for thousands of years, and a few scientific researchers (who also believe in that method) write some favorable research, even though the idea is far from a wide scientific consensus, some editors (who are also believers in that fringe idea) will point to that widespread use and that little bit of favorable research and claim a scientific consensus.

That's what I'm seeing here. Is that what you're trying to say? -- Brangifer (talk) 16:35, 20 July 2014 (UTC)

No, Brangifer, the relevant point is that the "widely accepted" phrase in FRINGE does not actually contain the qualifier, "by scientists". Cars are widely accepted as a form of transportation: We know that they are a widely accepted form of transportation because we know that there are a billion cars in the world. What scientists say about the validity of the automobile is irrelevant to its public acceptance. Similarly, TCM is a widely accepted form of health care: there are a billion people using it (at least sometimes) for health care. "Ineffective" is not the definition of "fringe"; "widely accepted by scientists" is not the definition of "widely accepted". WhatamIdoing (talk) 23:48, 20 July 2014 (UTC)

I'm not trying to make this out so it appears there is scientific consensus regarding the mechanism of acupuncture. I just want the article to be accurate and complete. Minority views should be indicated as such. - Technophant (talk) 16:48, 20 July 2014 (UTC)

I believe that's exactly the case already. -- Brangifer (talk) 17:24, 20 July 2014 (UTC)

+----------------------------------------------------------------------------------------------------+WAID - I think if there was a followup question to that audience about why they were taking supplements, I doubt any of them would say "to prevent cancer" or "to prevent aging" and bet that most would say "to ensure that I am getting the daily recommended amount of nutrients". The "why" is where the woo is, or is not. Yes?Jytdog (talk) 18:55, 20 July 2014 (UTC)

I suspect that each of them would have given you a different answer, but it is extremely unlikely that all of them would give what you've decided is the "right" answer, and even that "right" answer would be wrong, if you're eating a normal, upper-middle-class-vitamin-expert kind of varied, vegetable-filled diet.

I'm convinced that each of them would have been prepared to tell you why the research did not apply to their particular choice of vitamins: the Calcium came from chalk instead of coral, the Vitamin E in the study was cooked instead of raw, or whatever. (I don't suppose you've met anyone who has custom-made vitamin pills? I have.) WhatamIdoing (talk) 23:39, 20 July 2014 (UTC)

Interesting comeback! I guess neither of us know... (btw I didn't say "all" I said "most" :) and of course it is extremely unlikely that all of them would agree on anything! ) I wonder if the events of that story actually happened, or if it is just an urban legend. Jytdog (talk) 12:49, 21 July 2014 (UTC)

and the more i think about it, the more it has the hallmarks of urban legend. claims to be anchored in real events, but no date, time, or names to verify. shows "experts" being hypocritical (delightful!) and supports a position that "must be true" but is contradicted by mounds of published studies.... and for all those reasons, useful to deploy rhetorically. no slam on you intended, WAID, just thinking it through. so many of those floating around...Jytdog (talk) 13:38, 21 July 2014 (UTC)

Yes, it'd be fun to know if it really happened, wouldn't it? ("When" would be a particularly interesting point IMO.) But as the saying goes, "all the old stories are true--and some of them even happened!" WhatamIdoing (talk) 02:21, 22 July 2014 (UTC)

The problem is that the placebo effect is important here, alternative medicine works via the placebo effect while medical research is foccused on the effect a treatment has over and above the placebo effect. Count Iblis (talk) 19:15, 20 July 2014 (UTC)

This statement is not true. Homebirth is "alternative" in the US, and babies are not born by the placebo effect. WhatamIdoing (talk) 23:39, 20 July 2014 (UTC)

My 2 cents: the original poster has the same problem every POV pusher has; they come here with the expectation that the article should say something specific, and because the sourcing guideline doesn't allow them to say what they believe the article should say, they believe the sourcing guideline cannot possibly be correct because "it violates NPOV" ("NPOV" meaning in these cases what the POV pusher believe is true). This is, of course, the opposite of the appropriate way to determine what the NPOV is, which is to accurately summarize with due weight only those sources that meet our sourcing guideline. The first step in all cases is to find good quality sources based on our sourcing guidelines, and then summarize them to find the NPOV; determining what the POV should be first and criticizing the sourcing guideline because it does not allow that POV to be told is of course incorrect and should be rightly forbidden. Yobol (talk) 20:26, 20 July 2014 (UTC)

I believe in Wikipedia and following the policies and guidelines contained therein. I'm only trying, in good faith and through good practices, to help improve this article and in order to this there needs to be a change in this policy. In the binding arbitration Wikipedia:Requests_for_arbitration/Pseudoscience#Neutral_point_of_view_as_applied_to_science it's decided that

1a) Wikipedia:Neutral point of view, a fundamental policy, requires fair representation of significant alternatives to scientific orthodoxy. Significant alternatives, in this case, refers to legitimate scientific disagreement, as opposed to pseudoscience.

Now if Wikipedia requires fair representation but MEDRS disallows fair representation then MEDRS needs to be altered to allow ALL fundamental policies to be observed. - Technophant (talk) 20:40, 20 July 2014 (UTC)

You are missing the point. If there is a "legitimate scientific disagreement", it will be found in reliable sources, i.e. WP:MEDRS compliant sources. If you cannot find a view in high quality sources, there is no "legitimate scientific disagreement". NPOV comes from the sources that meet sourcing guidelines; it most certainly does not come from individual editor beliefs on what it should be. Yobol (talk) 20:54, 20 July 2014 (UTC)

Technophant, what Yobol said is dead on. You start with reliable sources and use them to ascertain the range of views and where they fall on continuum between consensus and FRINGE. There is no contradiction among policies and guidelines. A lot of smart and caring people have been working in WP for a long time and look out for things just like that. You are still not addressing the broader affect that what you are suggesting would have on the encyclopedia. To the extent you don't care, you are demonstrating that you are not here to build an encyclopedia. Jytdog (talk) 22:00, 20 July 2014 (UTC)

MEDRS is intended to deliver the highest quality medical information. Unfortunately, in areas where valid unorthodox opinions exist (like in Lyme disease), MEDRS tells editors to use the "highest quality" one, which is typically the orthodox one. This presents a false view of the topic. In any other non-medical article this wouldn't be a problem. The only place your are going to find both sides discussed is in newspapers or books, however they are not allowed as sources. - Technophant (talk) 22:19, 20 July 2014 (UTC)

Not responding to direct questions is a hallmark of tendentious behavior, Technophant. (please read that link and ask yourself if you are doing those things). I am done here, as you are just pushing your point, not conversing. (by the way, our article on Lyme disease controversy is well done and well sourced, IMO)Jytdog (talk) 22:58, 20 July 2014 (UTC)

It's not even that the highest-quality study happens to the orthodox one; the problem is that the POV pushers on the other side define the highest-quality study as being a source that is written, edited, peer-reviewed, and published exclusively by people who have solidly anti-altmed credentials and whose conclusion supports the anti-altmed POV.

According to one side, minority viewpoints deserve some space (see WP:YESPOV) to provide a reasonably complete description of their POV, including a description of claims that they make for scientific evidence. According to the other side, we shouldn't cite any sources written by professionals in that field, even if the source is a systematic review and concludes that the profession is not evidence-based medicine, and we definitely should not admit to even the possibility of the tiniest shred of evidence that something that is currently, according to our incomplete knowledge, considered to be "alternative" might have some possible efficacy.

I see no hope for resolution: we cannot re-write MEDRS to eliminate POV pushing in these articles, and we're getting it on all sides. WhatamIdoing (talk) 23:39, 20 July 2014 (UTC)

I agree that here is no hope for resolution on the sourcing guideline side. The editors need to work this out on WP:NPOVN or WP:DR. I agree that there are POV warriors on both sides. I have come to steer clear of most alt med stuff because of that. They make it impossible - and no fun - to get good, nuanced content into articles and stick. Jytdog (talk) 23:57, 20 July 2014 (UTC)

• Oppose change to guideline. No comment on the actual content being discussed here, other than to note that detailed arguments for a specific kind of content change for a specific article need to happen at the article Talk page or the WP:DR venue chosen. This spate of discussions raised in various locations smells of forum-shopping and points to possible disruptive editing, which I will investigate. Zad68 22:49, 20 July 2014 (UTC)
• Oppose change to guideline. Strong clear reliance on best sources possible and reflection of mainstream academic consensus of current medical knowledge should be maintained unequivicably in the guideline. Arguments based on the desire to change content in one article are not good basis to propose changing a guideline. Forum shopping, and tenditious IDHT is getting old. - - MrBill3 (talk) 06:42, 21 July 2014 (UTC)

+----------------------------------------------------------------------------------------------------+ For those who are not aware, the OP Technophant has been indef topic-banned from alt med and acupuncture topics, as per this ANI posting. Jytdog (talk) 22:49, 21 July 2014 (UTC)

src: upload.wikimedia.org

Two discussions of application of MEDRS to plant and herb articles in re alternative medicine

There are two related discussions of application of MEDRS in relation to plant and herb articles, in re choice of wording, efficacy, use, and usefulness , and traditional medicine and alternative medicine. One is at Wikipedia_talk:WikiProject_Plants/Template#Uses, the other at Wikipedia_talk:WikiProject_Plants#Plant_field_guides_and_manuals_do_not_meet_WP:MEDRS_standards_for_reliability_of_medical_information. FloraWilde (talk) 09:53, 22 July 2014 (UTC)

src: upload.wikimedia.org

Cannabinoid hyperemesis syndrome

It appears that Cannabinoid hyperemesis syndrome may be a medical article based on primary sources, contrary to the advice on this project page. But as a layman, I would like to hear from others better qualified to offer an opinion. Msnicki (talk) 22:51, 27 July 2014 (UTC)

src: upload.wikimedia.org

Why Not Primary Sources?

Primary Sources are sources in which the authors directly participated in the research or documented their personal experiences. They examined the patients, injected the rats, filled the test tubes, or at least supervised those who did. Many, but not all, papers published in medical journals are primary sources for facts about the research and discoveries made. That being the case, why shouldn't Primary Sources be referenced in articles about medical subject matter? Ormr2014 (talk) 17:35, 4 August 2014 (UTC)

This issue has been discussed a ton on this page - just search this page and the archive for "primary" and you will find tons of explanations. You can also see an essay I wrote called "Why MEDRS"? here. The very short answer is: 1) all WP content should be sourced from secondary sources as per WP:RS. Secondary sources tell us how much WP:WEIGHT to give information - it is beyond your authority or mine as editors here, to make those decisions. 2) More specific to the biomedical space.... Scientists publish their work to talk to each other; we happen to live in an era where we can witness that. But basic research in the biomedical space is exploratory - many hypotheses are tested and preliminary conclusions are drawn... and many of those turn out to false leads/dead ends. Experts in basic and clinical research sort themselves out - map where knowledge stands - in review articles (secondary sources). Those reviews tend to talk about what hypotheses are looking to be true and productive; they don't waste time documenting all the dead ends. The literature is littered with basic research papers that turned out to be dead ends (not retracted -- not egregiously flawed or fraudulent science - but just dead ends) and those papers just sit there in the literature. You or I have no way of knowing which have results that the field has found true and productive, and which are not. So we avoid them like the plague. 3) we are an encyclopedia, not a newspaper, and not a cutting edge journal. Our mission is to give reliable information to the public. So we don't add content based on the latest hyped-in-the-media paper, and we try to hard to only include content that is supported by experts in the field, reviewing the field - that is how we ensure the content is reliable. Jytdog (talk) 20:16, 4 August 2014 (UTC)

Re a recent change to wording / meaning

A change of wording (by NuclearWarfare) here that currently appears in the Biomedical journals subsection seems to me to be significant. The change is from

Research papers that describe original experiments are primary sources; however, they normally contain previous-work sections that are secondary sources (these sections are often incomplete

to

Research papers that describe original experiments are primary sources; however, they normally contain sections that place their research in the context of previous work. These sections may be considered secondary sources for the purpose of this guideline, but note that these sections are often incomplete

The current wording seems to me to imply that such content may be considered an acceptable secondary source under certain circumstances. I feel the wording and positioning of this statement needs some attention to avoid unintended interpretations. So I made this edit for the sake of clarity, which hasn't actually survived (just for the record, backstory details here). Either way, such changes to the wording seem to me to have some substantial implications, and probably require consensus (or otherwise) to be reached here.

Fwiw, I don't have strong feelings on this, though I too would tend to support a flexible approach in such situations (I think). 86.128.169.211 (talk) 19:12, 29 June 2014 (UTC)

NuclearWarfare is correct: previous-work sections (that exceed a couple of sentences) are (usually) secondary sources. (The presence of analytical statements is key.)

The thorny question is really whether we want to advertise this fact to editors. Advertising it to editors like NuclearWarfare would be an excellent idea. NW's groks due weight and is already attuned to the risks of using these sections. Advertising it to a very inexperienced editor or a POV pusher would probably not be such a good idea. We may end up with refspam, creeping reliance on primaries, and cherry picking. I don't know what to recommend here. WhatamIdoing (talk) 19:53, 29 June 2014 (UTC)

Per WAID's considerations, I had two main thoughts about how we might handle the actual wording:

• that it's probably better to discuss use of research papers after (rather than before) talking about reviews;
• it's perhaps a good idea to specify rather clearly the sort of situation in which it's appropriate to source information contained in a research paper.

In this version (diff here) I felt I was probably interpreting both NW's thoughts and broader consensus (though I accept I should probably have discussed here before making the actual change). 86.128.169.211 (talk) 20:27, 29 June 2014 (UTC)

WhatamIdoing, maybe one thought would be to add in a stipulation that in the cases of contentious topics, primary sources are a no go when even citing the introduction as a form of secondary literature? That way we could allow primary literature introductions in cases where it's something that should be cited from a journal, but isn't anything likely to be in an area where a POV pusher could really do damage. If I'm citing a paper that lists in its introduction what pathogens penicillin has been used against, that would seem like a non-issue, but cherry picking a fringe paper that mentions a link to autism with vaccinations in its introduction would definitely be a cause for concern. So basically say primary sources can be used with caution in this sense, but if anyone raises a fuss, it's gone.

This is just a potential solution, but determining what would be a contentious topic would be subjective, so I'm not sure how feasible the idea would be. This could allow some flexibility where we can error on the side of caution, and just say that if someone at all disputes the addition of a primary source's content, it's then contentious and shouldn't be used. If we're going to have some ambiguity in the guideline, I'd at least say have that wiggle room skewed towards the use of secondary sources rather than make it easy to put in primary sources. Personally I'd prefer just secondary sources for the reasons you mentioned above, but maybe this approach might work? Kingofaces43 (talk) 18:28, 14 July 2014 (UTC)

That's an interesting way to address the problem. I think it would work, at least if people are challenging it for actual/specific reasons, rather than the sort of "I challenge every sentence followed by a primary source from here to infinity" thing that we see every now and again. It's not very different in spirit from the line that we use at WP:ELBURDEN, which is that any external link that's is disputed should be removed: assume that it's fine, remove if there are indications that it's not. However, it's a bit more complicated, given that we're supporting article information. A weak source is sometimes better than no source.

NuclearWarfare, what do you think? Do you think that there's a risk of this being gamed? I'm a little bit worried about people adding REFSPAM, and moderately worried about people fussing that the source isn't good enough, but not being willing to replace it with a better one themselves. WhatamIdoing (talk) 22:44, 14 July 2014 (UTC)

Grok, now there's an interesting phrase that I haven't heard of before. I like it.

Anyway, as to the point raised by Kingofaces43 (had a hard time not reading that as "King of faces," which weirded me out a little): it's a defensible position. And I know how easy it is for "citogenisized" mistakes to spread even just within in the professional literature, because peer reviewers and editors often don't seriously review the background sections. But I think it's taken a little bit far. I just added some content to the [arthroplasty]] article last week that summarized the latest state of the meniscectomy literature. For that, I largely relied on the background sections of the major RCTs that have been published in the area (all in either the major general medical journals or the major speciality journals, so high quality sources). This information simply hasn't been well synthesized in reviews yet. It's a little too new (~seven years since the original trials started being published) and a little too controversial (partly because the observed data has high variance) for treatment guidelines. But I think it is still important for Wikipedia to reflect the consensus in the broader field. This change would take it a little too far in prohibiting that, at least as I read it. How about something like the following: Research papers that describe original experiments are primary sources; however, they normally contain sections that place their research in the context of previous work. These sections may be used in Wikipedia with care. As they are often incomplete, such citations challenged in good-faith by another editor should be replaced by a more comprehensive source [if possible?]. I'd welcome input. NW (Talk) 01:42, 15 July 2014 (UTC)

I would amend it to Research papers that describe original experiments are primary sources; however, they normally contain sections that place their research in the context of previous work. These sections may be used in Wikipedia with care. As they are often incomplete, such citations challenged in good-faith by another editor should be removed or preferably replaced by a more comprehensive source if possible. Verifiability and reliability of content are what makes up core policy for Wikipedia's content standards, so I'd rather see it be harder to add information from primary sources (although still possible). The purpose of this set up is to raise the bar of reliability for any content or topic considered contentious (which seems like it should be the case anyways), and to allow the possibility of primary sources in less contentious topics where there likely wouldn't be a dispute anyways.

What I'm not sure about though is how this would affect the quality of articles depending on how active editors are. In other words, would it be easier for poor quality content and sources to creep in for lesser known topics, or is this happening already since they aren't checked as often anyways? This approach would basically endorse that we accept lower quality evidence for some articles, but require more for others. I'm a little split on that thought. I think this is good in contentious topics so that we can stick to high quality sources and avoid a lot of the issues that come from POV, reliability, and other common problems in those discussions, but I'm not so sure about essentially endorsing what seems like a lesser standard for potentially less checked topics. Maybe it's not really that different than what typically goes on now though? Kingofaces43 (talk) 15:08, 15 July 2014 (UTC)

@Kingofaces43: You're right that it's always best to use the highest quality sources possible, even for lower profile articles, but part of the consequences of certain topics being lower profile is that the highest quality sources don't always exist. I think that your phrasing is fine. Perhaps we can do more to emphasize that we really do mean that the challenge must be in good faith as to the editor's concern about The Truth^(TM) factualness of the statement. NW (Talk) 21:30, 19 July 2014 (UTC)

If you two wanted to take a baby step towards this, rather than plunging forward (which I'm not opposed to), then you could limit its application to "subjects, such as rare diseases, for which review articles and reference works are not abundant". Alternatively, you could try it out for a while and see what happens. We can always remove this statement if it proves to cause problems.

In response to a comment above, there are types of basic information that people expect an article to contain (e.g., basic information about signs and symptoms, epidemiology, diagnostic process, typical treatment), regardless of whether that information happens to be present in a review or reference book that was published in the last five or ten years. WhatamIdoing (talk) 23:48, 19 July 2014 (UTC)

Sorry I'm a little late in responding WhatamIdoing, but in a case of basic information like you describe, wouldn't that then be something unlikely to be contested? The type of content you're describing sounds like it wouldn't be contentious at all, and if someone did raise a fuss about the sourced content (then showing there is some contention) then the editors would have to go to more reliable secondary sources as they probably would do anyways to show the information is indeed factual and not some artifact from being in the primary literature. It seems like a fluid way to handle the situation, so I'm curious where you see an issue in your example? I'm not seeing one, but if you could explain what you're thinking of a bit more maybe we can figure out a way to tighten up the language a bit more. Kingofaces43 (talk) 16:32, 30 July 2014 (UTC)

Even if the specific information is unlikely to be challenged, good editors are usually dissatisfied with their work if there are zero sources in an entire section. WhatamIdoing (talk) 16:29, 11 August 2014 (UTC)

Cancer Research UK Cancer Stats?

Do medical editors here have any particular views on the appropriateness or otherwise of using this internet service as a reliable medical source, especially when recent pertinent MEDRS are hard to come by (e.g. for coeliac disease as a putative risk factor for esophageal carcinoma - [11] vs. [12])? Although their Cancer Stats parent page gives the impression of being only UK, in reality this internally researched source of public information also provides some more global content. (ping Wiki CRUK John to alert him of this query) 86.157.144.73 (talk) 15:26, 11 August 2014 (UTC)

The service is reliable enough, but Wikipedia does not present statistics unless those statistics are themselves reported by a secondary source. There are some exceptions which are commonly reported in infoboxes. You are proposing to reference interpretations of the statistics. In my opinion, these kinds of states are often useful, but I have never seen these kinds of statements made often on Wikipedia so there may be other opinions if anyone tries doing this. I like the idea. Blue Rasberry (talk) 18:09, 2 September 2014 (UTC)

I don't see any inherent barrier to accepting these as valid MEDRS and using them ad lib. This is a large national organization and if we can reference one of their reports all is good. -- CFCF ? (email) 05:46, 4 September 2014 (UTC)

The pages are nicely referred and if supported by a secondary source we can just use that. It gives us a better idea of the data of publication of the meta analysis or review in question. NCI is a decent source. Doc James (talk · contribs · email) (if I write on your page reply on mine) 06:41, 4 September 2014 (UTC)

Acupuncture source

Seeking comments re a 2011 review of reviews by Ernst (PMID 21440191). (This may have come up here before; I know it came up someplace, but I can't remember where.)

The debate is about how to interpret a particular sentence in context. It boils down to a right and wrong way to read a source and unfortunately requires more than just a glance (but not much more than, say, 10-15 minutes, depending on one's background). I've outlined the issue at User:Middle_8/Ernst-11. That should provide enough background, but I highly recommend also reading the paper if possible. Feel free to email me for a copy.

Simplest summary yet: User:Middle_8/Ernst-11-excerpts --Middle 8 (contribs o COI) 18:44, 6 September 2014 (UTC)

Note: Unfortunately, there's been some heat over there; hoping for lots of light! (And also hoping to keep this focused just on content and not on personalities.) Thank you very much for your time! --Middle 8 (POV-pushing o COI) 17:55, 2 September 2014 (UTC)

We've been through this before. I'd urge anyone interested to read Middle8's commentary at User:Middle_8/Ernst-11 and try to understand the point he's making: that Ernst has reached a conclusion (real and sham acu are equally effective), but because he refers to that conclusion in the context of some primary sources that he has reviewed, that means that we can't use the conclusion he states. This is completely against our policy of respecting secondary sources; none of us can know how many sources, primary and secondary, he has examined to reach his conclusion. The primaries he refers to are introduced specifically to refute the efficacy of acu in treating LBP and osteoarthritis (the only areas where his review of secondaries suggested showed efficacy beyond placebo), and we cannot baldly assume that these were the only sources that contributed to Ernst's stated conclusion. How many times do we need to state that Wiki editors are not qualified to interpret sources? - and that's why we are so insistent on using secondary sources for ourselves; we cannot make judgements on a respected author's choice of sources because his expertise is what we rely on. We should never be trying to substitute Middle8's interpretation of secondary source for what the source actually says.

We all know that 'real' acupuncture can have positive effects on pain; but we also know that there is no credible mechanism that could produce different results from 'sham' acupuncture. That the two are equally effective within the bounds of experimental error is obvious to anyone who does not believe in "qi" and "meridians" (in other words, the whole scientific community) and we should have no difficulty in stating that clearly within the article. --RexxS (talk) 18:46, 2 September 2014 (UTC)

A simpler summary: the question is, should we cite Ernst's statement "real acupuncture was no better than sham" as being the a conclusion of his review per se, or not. The statement appears in the first paragraph on p. 762 of PMID 21440191.

In context, it is apparent that his statement refers to some recent, high-quality trials that showed null results. Contrary to RexxS's reading, it does not refer to the reviews he looked at, and is not a general conclusion of his review. The majority of those reviews were actually positive for acu, but with numerous contradictions and caveats. That's why he said those result should be seen in light of recent, better trials. The highlighted text at User:Middle_8/Ernst-11 illustrates this context, and the unlikelihood of RexxS's reading. --Middle 8 (POV-pushing o COI) 05:32, 3 September 2014 (UTC) ..... edited: "the conclusion" ==> a conclusion" 23:51, 4 September 2014 (UTC)

IOW, when Ernst says "real acupuncture was no better than sham" refer to, which literature exactly is he referring to? I agree with 2/0's conclusion [13]. --Middle 8 (contribs o COI) 02:19, 6 September 2014 (UTC)

And of course he also says, as a general statement: "Adequately controlling for nonspecific effects in future is likely to demonstrate that acupuncture has no or few specific effects on pain." That's his take on "the forest", while Suarez-Almador is one of the "trees" (and one that happens to represent the "forest" pretty well). --Middle 8 (contribs o COI) 02:27, 6 September 2014 (UTC)

Here was the response to 2/0's conclusion. QuackGuru (talk) 02:30, 6 September 2014 (UTC)

It's clear to anyone who reads the article that "individualized acupuncture is not better in reducing symptoms than formula acupuncture or sham acupuncture with a toothpick that does not penetrate the skin" is Ernst et al.'s interpretation of Cherkin et al. (2009) and should only be quoted as such. The conclusion of Ernst et al. is the one in the abstract "In conclusion, numerous systematic reviews have generated little truly convincing evidence that acupuncture is effective in reducing pain. Serious adverse effects continue to be reported." Whether Ernst et al. were justified in reaching their conclusion isn't for us to evaluate in article space, whatever our personal opinions. Peter coxhead (talk) 08:40, 3 September 2014 (UTC)

While not the primary conclusion of the review of review (the study does not appear to be designed to answer the question of sham vs real and so cannot be said to be the primary conclusion), Ernst spends an entire paragraph (the largest paragraph in the discussion section, as a matter of fact) to basically endorse two high quality primary studies finding "real" wasn't different than "sham". My interpretation: It would be incorrect to say that discussion about sham acupuncture was the only or primary conclusion of the paper; however, does seem to be a conclusion Ernst is making and therefore citable as a source for that conclusion. Yobol (talk) 13:57, 4 September 2014 (UTC)

Meta comment: I saw a research report recently that said the involvement of experienced editors led to less collaborative editing in sexuality-related articles. I wonder if the same would be true for altmed. WhatamIdoing (talk) 01:13, 3 September 2014 (UTC)

Scary. I think alt-med wars are a partisan conflict that transcends experience level. Then again, experience might make it worse, insofar as it leads to more creative and subtle ways of being tendentious. In fact, there are some really good editors who can and do collaborate very well, but are discouraged by the tendentiousness. However, they can't always agree among themselves who are the worst offenders, so....

This reminds me of a comment I heard about why academics argue so intensely among themselves: because the stakes are so low (sic). "Someone is wrong on the internet." Arguing becomes a recreational activity, a way to blow off steam.--Middle 8 (POV-pushing o COI) 05:22, 3 September 2014 (UTC)

JzG/Guy has written or will write to Ernst himself to clarify, which should settle the issue, as long as it's handled precisely and transparently -- but this broad-brushed and dismissive comment does not exactly inspire confidence. I look forward to hearing the specifics. We can get both the letter and spirit of the law right without doing violence to either. That's the hallmark of a good encyclopedia. --Middle 8 (POV-pushing o COI) 05:04, 4 September 2014 (UTC) edited 07:06, 4 September 2014 (UTC)

I don't see how that will help at all. A response from the author by e-mail is irrelevant; it can't be used here. Our task is solely to represent, accurately, what the published source says, which is crystal clear in my view. Peter coxhead (talk) 08:56, 4 September 2014 (UTC)

If the author replies that it obviously says whatever it is that you believe it says, then it could help quite a bit. Also, if the author replies that he meant exactly the opposite of what you believe it says, then that, too, should help quite a bit. When people disagree about "what the published source says", then it's helpful to get expert opinion on the question of "what the published source says". The author of said source is a reasonable expert on the question of what he intended to communicate in that source. WhatamIdoing (talk) 15:20, 5 September 2014 (UTC)

But how can we use an author's reply? In a scientific paper, we could put "pers. comm." but not in Wikipedia, so there's no way of sourcing the author's interpretation. Peter coxhead (talk) 15:40, 5 September 2014 (UTC)

If the author's reply is that "I meant X, not Y", then it stops the argument amongst editors where some say "the author meant X" and others say "the author meant Y". We can use it to inform how we report the source and it does away with all the nonsense of trying to second-guess what was in the author's head at the time. When we go through the process of identifying reliable sources, we don't have to cite our discussions on talk. In the article the citation is still to the source, but at least we know when an incorrect spin was being placed on it. --RexxS (talk) 19:01, 5 September 2014 (UTC)

But how is the reader supposed to know that the source is reported correctly, if it's not obvious from the reading the source? If it's necessary to go to these lengths, it's clear that something is wrong in the first place. Peter coxhead (talk) 19:47, 5 September 2014 (UTC)

The end of that line of argument is "We must never use any source unless all editors (including POV pushers and people with too little education in the area to understand the source) agree on what the source says". Our job is to accurately summarize sources, not to help readers (or editors, for that matter) who have difficulty understanding the original source.

If Ernst replies and says, "That entire paragraph is all about that one study", then we should not use that one sentence as if it were a conclusion about all studies. If Ernst replies and says, "That paragraph happens to mention one study, but the sentence in question is actually about all of them", then we can use that sentence as a general summary (or at least consider using it). WhatamIdoing (talk) 03:55, 6 September 2014 (UTC)

I don't agree that it's the end of that line of argument; this is an encyclopedia and we expect appropriate standards of understanding by editors (and readers). Asking editors to reach consensus does not mean that all have to agree "including POV pushers and people with too little education in the area to understand the source". I understand only too well that it's hard to reach consensus in this area, but that's what editors must try to do, remembering that "consensus is ascertained by the quality of the arguments given on the various sides of an issue, as viewed through the lens of Wikipedia policy."

As to any reply from Ernst (who, note, is only the first author, not the sole author), we will have to differ on whether it can influence the content of the article. Peter coxhead (talk) 06:09, 6 September 2014 (UTC)

You can differ all you like, it doesn't alter the fact that sources are cited in the article to provide verifiability and a basis for further reading, but choosing them is done by the editors, not the readers. Consumers want to eat the sausage, not be given an account of how it was made. --RexxS (talk) 11:35, 6 September 2014 (UTC)

(Middle 8 asked me to comment on this point; I am active at the Acupuncture page but had hitherto been avoiding the relevant talkpage discussion as having a poor heat to light ratio). Middle 8's reading of the source in context agrees with mine that the sentence "Real and sham acupuncture were both more effective in reducing pain than no acupuncture at all, but real acupuncture was no better than sham" is a report of the conclusion reached by Suarez-Almazor 2010. Ernst 2011 is pretty harsh on the quality of available evidence and should by no means be cited as supportive of the practice, but this particular statement is providing context for the results being reported: it is not a result itself of the present work. Being cited in this way does, however, provide a good argument for citing Suarez-Almazor 2010 itself for this conclusion (limited to knee osteoarthritis if we want to be very particular). - 2/0 (cont.) 15:12, 4 September 2014 (UTC)

To clarify, we do cite the systematic review 2011 not the older Suarez-Almazor 2010 source for the text. We usually don't use randomized controlled trials (RCTs) as good MEDRS. We typically use the reviews who cite the RCTs and/or other sources. QuackGuru (talk) 15:38, 4 September 2014 (UTC)

@QuackGuru: Are you agreeing that Ernst's statement "real acupuncture was no better than sham" can be cited for Suarez-Almazor 2010 and only for Suarez-Almazor 2010? If you and other editors agree, then we can close this. (In no way to I dispute what Ernst's view of the bigger picture is). --Middle 8 (contribs o COI) 07:44, 6 September 2014 (UTC)

That would suggest that we could take apart a secondary review and eventually attribute authors' conclusions to the primary studies that we think influenced their statement. That is precisely what we don't do in identifying reliable sources for medicine; that sort of deconstruction is the purview of other published articles, not the amateur detective work of Wikipedia editors. --RexxS (talk) 11:24, 6 September 2014 (UTC)

Ernst '11 (PMID 21440191) isn't a review of primary studies. --Middle 8 (contribs o COI) 12:59, 6 September 2014 (UTC)

Indeed it isn't. But if you can do it to that secondary source, then you you can do it for any secondary. Once you start down the slippery slope of substituting your own analysis for that of the published author, you end up with articles filled up with editors' POV based on the weakest primary sources. --RexxS (talk) 18:11, 6 September 2014 (UTC)

Ironically, it sounds like we agree on sticking close to the author's analysis. But to do that, we can't just take a given statement from the article and say "Ernst said this". We have to account for context. --Middle 8 (contribs o COI) 05:13, 7 September 2014 (UTC)

Cinnamon and WP:MEDANIMAL

I cleared out a bunch of primary sources from this article (which said Cinnamon has "anti-viral" properties against HIV, treats Alzheimer's in mice, etc.) but they have been re-instated, on the basis of WP:MEDANIMAL. In general, although WP:MEDANIMAL is quite carefully written, it does look a bit like it "opens the door" against our general wariness of primary sources by saying "Where in vitro and animal-model data are cited ..." without making it crystal clear where that may be appropriate. Is this an issue? Alexbrn ^{talk|contribs|COI} 05:44, 5 September 2014 (UTC)

The rest of that sentence appears clear that such a use would not be OK without an appropriate disclaimer. Maybe just a little extra emphasis there would help, e.g. "Where in vitro and animal-model data are cited on Wikipedia, it should be clear to the reader that the data are pre-clinical, and the article text should avoid stating or implying not state or imply that the reported findings necessarily hold true in humans. The level of support for a hypothesis should be evident to the reader." --Middle 8 (contribs o COI)

I think the problem might hinge on weight: it's not okay to give an article a "Medical research" section, and then incautiously pile primary sources into it unless some secondary discussion gives those sources some weight. So the section at Cinnamon is undue in my view (An in vitro experiment in 2000 found anti-viral properties ...?). And in this case the editor has been encouraged by WP:MEDANIMAL. Alexbrn ^{talk|contribs|COI} 08:08, 5 September 2014 (UTC)

Alexbrn your edits were dead on. Am watching the page now. Jytdog (talk) 11:21, 5 September 2014 (UTC)

Using WP:MEDANIMAL and taking it out of context of the rest of MEDRS seems to be the problem here. Those sources were still primary sources (fails WP:MEDREV) and out of date (fails WP:MEDDATE). This guideline has to be taken as a whole, not saying "It satisfies one paragraph, I can ignore everything else!". Agree with the removal of that content. Yobol (talk) 15:10, 5 September 2014 (UTC)

Agree with all of the above... good edits... do you think the guideline is clear enough as is? Experienced editors will know how to address problems, but I wonder if making it more explicit might prevent mistakes by less-experienced editors? No strong feelings either way; just a thought. --Middle 8 (contribs o COI) 13:28, 6 September 2014 (UTC)

Anything that reduced the number of mistakes made by editors in failing to assess WEIGHT in the context of MEDRS would make life far easier for the regulars at WPMED. I'd certainly support extra clarity, but the devil, of course, is in the detail: it's always a problem to strike a balance between guidance that becomes over-prescriptive and guidance that leaves itself open to too much interpretation to be useful.

One of the ways that editors found of explaining NPOV is by producing a FAQ - it's at Wikipedia:Neutral point of view/FAQ and expands on the NPOV guidance in a very readable manner. I wonder if making a FAQ for MEDRS might be a way forward? --RexxS (talk) 18:27, 6 September 2014 (UTC)

That is a really good idea. It's much better to have the guideline itself be concise. And there's no shortage of FAQ-ish material in the archives etc. ... feel like starting a draft? --Middle 8 (contribs o COI) 03:27, 7 September 2014 (UTC)

Jytdog has started something like that if you haven't seen the essay yet (https://en.wikipedia.org/wiki/User:Jytdog/Why_MEDRS%3F). Still a work in progress from my understanding. Kingofaces43 (talk) 01:47, 8 September 2014 (UTC)

We'll need something shorter, I think. I've started Wikipedia talk:Identifying reliable sources (medicine)/FAQ for you. Feel free to blank it and insert whatever you like. In general, my experience is that shorter lists covering only frequently asked questions are more useful than lists that try to be comprehensive. I've been pretty satisfied with the results of the FAQ at the top of WT:RS and WT:V (it's the same FAQ on both): the frequency of certain questions dropped significantly after posting them. So I'd suggest the top three or four questions, and we can add more later if we need to. WhatamIdoing (talk) 01:18, 29 September 2014 (UTC)

The key is the use of high quality secondary sources to put research into context and to give it an overview. Review articles do sometimes comment on primary sources based on animal studies. Doc James (talk · contribs · email) (if I write on your page reply on mine) 02:32, 29 September 2014 (UTC)

Different types of meta-analysis

RE this undo [14] (ping CFCF): the sentence in question currently speaks of "meta-analyses of randomized controlled trials". Don't RCTs, by definition, regard efficacy of an intervention (of whatever sort)? 109.153.156.71 (talk) 18:33, 16 October 2014 (UTC)

I was a little quick and misinterpreted which section this was added to. I've restored your edit. -- CFCF ? (email) 19:41, 16 October 2014 (UTC)

Thanks CFCF. Of course there are other relevant types of meta-analysis too... 109.153.156.71 (talk) 20:01, 16 October 2014 (UTC)

RCTs can report other findings such as adverse events, so we probably shouldn't limit the language. Yobol (talk) 20:19, 16 October 2014 (UTC)

Meta-analyses of RCT generally aim to provide "best evidence" of efficacy, so I think it is appropriate [though not something worth getting into any sort of a conflict over] to let readers know what we're referring to there (or else rephrase the sentence). Yobol, can you provide an example of a meta-analysis of RCTs primarily designed to investigate harm? 109.153.156.71 (talk) 21:30, 16 October 2014 (UTC)

RCTs provide information other than primary end points (such as adverse events) and meta-analyses of these events can be good sources of information about these (see for example this study). Meta-analyses can get good data from RCTs on other aspects of therapy than just efficacy. Yobol (talk) 22:01, 16 October 2014 (UTC)

Agree. Discussion closed :) 109.153.156.71 (talk) 03:28, 17 October 2014 (UTC)

Primary sources "worth mentioning"

I know in recent months some progress has been made to made MEDRS clearer in its general prohibition of using primary sources for health content (except in exceptional circumstances) but I think there is still some distance to travel. At Homeopathy an editor is arguing that the wording of WP:MEDPRI gives them permission to use a primary source.[15] Furthermore I wonder here about the wisdom of mentioning that a primary may be worthy of use because it is "surprising" - mightn't that rather indicate we should be cautious about it? Alexbrn ^{talk|contribs|COI} 15:32, 11 October 2014 (UTC)

i would think so! this is exactly what medrs-compliant WP:SECONDARY sources are for - to put new research findings in perspective. judging the importance/surprising-ness of results published in a primary source is OR in my view. Jytdog (talk) 15:44, 11 October 2014 (UTC)

I have to agree also. "Surprising" results (i.e. ones that are out of line with the rest of medical literature) should be precisely the type of primary study we should NOT be using. Yobol (talk) 15:48, 11 October 2014 (UTC)

Some surprises are of the kind where a new key observation or insight makes sense of known puzzles, such as for H. pylori or DNA. These incidentally overthrow a body of unsatisfactory "best guesses" and meet resistance because of that. Such cases characteristically win major notice in secondary sources (even in Stockholm) and so only need MEDPRI exception for a very short time. The case at hand on Homeopathy is nothing of the sort and is in fact a simple case of cherry-picking one of many similar studies to get the "right" result. We should be able to make a clear and succinct explanation that does not serve to encourage this nonsense.-- Preceding unsigned comment added by LeadSongDog (talk o contribs) 18:44, 11 October 2014 (UTC

problematic but perfect examples. :) the notion that H pylori caused ulcers was considered ludicrous for a long time after it was initially proposed. that is actually a great example of why we should stay away from primaries. none of us have the authority to be arbiters of what a given field holds as true, and sometimes that means WP may be "wrong" for some time, until the field changes its mind. in your other hand, the publication of the double helix was pretty much immediately hailed as the right answer.Jytdog (talk) 19:19, 11 October 2014 (UTC)

The problem is that for every H pylori, there are hundreds or more primary studies that will have no effect whatsoever on the literature in the long term, especially the "suprising" ones that receive media attention briefly but fade away. If we're going to keep a discussion about surprising results, we probably need a better way of separating the wheat from the chaff. Yobol (talk) 00:17, 12 October 2014 (UTC)

i think we are all singing from the same songbook so far. :) Jytdog (talk) 00:32, 12 October 2014 (UTC)

Not only are there a hundred nonsense study for every groundbreaking one, encyclopedias are not the place for promoting the latest potential discoveries. There are plenty of other venues for that, WP is an encyclopedia. - - MrBill3 (talk) 07:15, 13 October 2014 (UTC)

+----------------------------------------------------------------------------------------------------+While I would agree certain primary sources are very influential and important the issue is also one of who should make such a judgment. The benefit of having a guideline which gives a clear demarcation is that we don't need to waste time deliberating whether an article can be included or not. Groundbreaking articles will often be reviewed exceedingly fast, and can be included quickly anyway, and if per the odd chance they were "noticed by Stockholm" before they are reviewed that "notice" is strong enough to bear a mention in the article anyway, but on different grounds ("reputable major medical and scientific bodies"). -- CFCF ? (email) 11:03, 13 October 2014 (UTC)

I've been mulling over a very similar topic, so I thought I'd pose it here and see if it helps identify what we should be looking at in the above. When we think about why we use MEDRS as a guideline, why is that? One reason is that we especially want to be sure health related content is correct (or as good as we can get it) because people do use that information for health research (whether that is good or bad is a topic for another time). The other comes from those of us more experienced in scientific literature where we know that primary literature can be often incorrect and intended to be a very fluid search for information among scientists specifically. The general public (and many editors) have trouble with that concept and sometimes hold content in primary sources to a much higher standard than science those with research experience would.

So with those two main very related reasons for why we avoid primary sources, is one more at the core of MEDRS than the other in some editors' minds? Are we reaching for secondary sources primarily because we understand how scientific literature works, or because we consider health related content really important? It's nuanced, but I get the feeling editors might make slightly different interpretations depending on which perspective they align with a little more. If you get the importance of health content, but don't know the science literature background, primary literature might not seem so horrible. If you come from the scientific literature background (as a scientist this is where I sit more) you know very well why primary literature has issues and find it very difficult to find a reason to use a primary source. I'm betting more people don't have much background when it comes to scientific literature, so is that what we're mostly dealing with when people are trying to grapple with MEDRS? Anyways, that's just my two cents on potential editor perspectives to consider when tightening up MEDRS. Maybe there are more things we could specify to help folks out, but it could be something we can't necessarily contend with within the guideline itself much more either. Kingofaces43 (talk) 03:20, 14 October 2014 (UTC)

I'd say the nature of research publishing provides the reason why primary sources shouldn't be used, since the aim is for Wikipedia to "accurately reflect current medical knowledge", as MEDRS puts it. MEDRS is currently good and makes sense overall, but I think needs to be written more defensively in some places to prevent misunderstanding from editors who won't read all of it and who lack an understanding of the sourcing issues. The section under discussion(WP:MEDPRI) is an example of where this 'defense' needs to be strengthened. Alexbrn ^{talk|contribs|COI} 04:08, 14 October 2014 (UTC)

I think that if we go much further in "banning" primary sources, that we're going to have problems with rare diseases and possibly a revolt from the rest of the community.

What might be more productive is to create a WP:MEDDUE parallel to WP:DUE. We're not avoiding primaries because they are almost always wrong. We're not avoiding primaries because they're aimed at other fact-seekers. Primary sources are absolutely reliable sources for certain, very limited statements. In between, say, when the Women's Health Initiative trial for hormone replacement therapy was stopped and when that information turned up in independent secondary sources, it would have been perfectly reasonable to make a statement--even a statement sourced to a press release or a newspaper article--that "This trial was stopped early because it appeared that HRT caused heart disease rather than preventing it". Nothing short of a solid secondary source, however, would have been strong enough to make the statements now in the relevant articles, namely that HRT causes heart disease. Whether a source is reliable depends on exactly the words you want to support with that source.

So why are we avoiding them? We're avoiding them because they're easily misrepresented, overstated ("HRT causes heart disease"), and cherry-picked. The main problem is one of WP:DUE weight, not primarily reliability. WhatamIdoing (talk) 04:48, 14 October 2014 (UTC)

This is a great idea. Would such a discussion belong under the "respect secondary sources" section, or go elsewhere? I think this would clear up a lot of confusion novice medical editors have about why we frown on most usage of primary studies. Yobol (talk) 20:09, 17 October 2014 (UTC)

Under ==Respect secondary sources== might work, but I've been thinking about putting it on a separate page for a while. WhatamIdoing (talk) 03:17, 19 October 2014 (UTC)

Medical and scientific organizations

I have a question to clarify in my mind something that is not expressly addressed. Medical and scientific organizations may have websites. On these may be FAQ's and other opinion pages that dont cite who wrote them, or the sources that went into conclusions they make or answers they give. Are those pages medical reliable sources? I dont believe they are but just want to make sure. AlbinoFerret (talk) 03:11, 14 October 2014 (UTC)

Websites of scientific organizations of undisputed reliability (such as the CDC website, to name one I consulted recently) often don't have specific authors or specific references. So, I would say, yes, these websites can be reliable even without specific author information or specific citations. Yobol (talk) 03:31, 14 October 2014 (UTC)

@AlbinoFerret: It would depend on the details, but my inclination would be to say FAQs were generally reliable, at the very least for relaying what that organizations's view was. Alexbrn ^{talk|contribs|COI} 03:36, 14 October 2014 (UTC)

@Alexbrn: Thanks for that answer, so while it is a reliable source, WP:WEIGHT would come into play on if, how, and where its used in the wikipedia article. AlbinoFerret (talk) 22:34, 14 October 2014 (UTC)

Again, it would depend. Weight is nearly always a consideration but I'd have though if a major medical organization had thought to include something in a FAQ, it'd be a pretty good indication of weightiness. Alexbrn ^{talk|contribs|COI} 05:03, 15 October 2014 (UTC)

In that case another reference shouldnt be hard to find and used instead of a Facts page. Relying on the facts page and refusing to provide another reference is a red flag to me on WP:WEIGHT. The reference may be used, but it would have to be attributed to the organization instead of wording that would make it look more widely accepted if just the one reference can be found. AlbinoFerret (talk) 19:41, 17 October 2014 (UTC)

It appears to me you asked general questions (such as "can pages without authors or citations be MEDRS compliant"; the answer is yes it may be reliable, but it is dependent on the specifics). Whether any specific page is reliable for specific content is case dependent. You appear to have a source in mind, and seem to be arguing about it here without providing any specific guidance as to the specifics, which is beyond the scope of this particular talk page. Yobol (talk) 20:07, 17 October 2014 (UTC)

Here is the page. AlbinoFerret (talk) 02:36, 18 October 2014 (UTC)

The relevant text is "The National Institute on Drug Abuse stated that there is a possibility that they could promote continuation of addiction to nicotine in those who are attempting to quit.[43]" See Electronic_cigarette#Addiction. QuackGuru (talk) 02:45, 18 October 2014 (UTC)

Which you placed in the lede ( a prominent position for a single reference source from a facts page) and would not find another reference that said the same thing when asked a number of times. The original edit by QuackGuru was "They may promote continuation of addiction in those who already smoke" The form he posted didnt come about until it was moved down the page and edited days later when QuackGuru didnt find another reference to back up what he placed in the article. AlbinoFerret (talk) 03:04, 18 October 2014 (UTC)AlbinoFerret (talk) 02:55, 18 October 2014 (UTC)

AlbinoFerret, once an editor has provided a source that he honestly believes is sufficient for the statement being made, then his WP:BURDEN has been met.

I notice that you said that he "didn't" find a stronger source, rather than he "couldn't" find a stronger source. If you know of a stronger source, then you are free to upgrade that source whenever you want. If you believe it likely that a stronger source could be cited for the same material, then it would be disingenuous to fight the content based on the existing source. WhatamIdoing (talk) 03:22, 19 October 2014 (UTC)

I just found another source and I added the text to the lede. QuackGuru (talk) 05:33, 19 October 2014 (UTC)

WhatamIdoing I could not find another source, I would have replaced the source if I had found one. At first I thought the page was less that reliable but found it was a weight problem problem. After looking I left it up to the person who placed it there to find another. It was removed from the lede a few days later when he didnt find a source, and placed in the body. So the information stayed in the article. The problem wasnt with the information, but finding a way to attribute it properly. AlbinoFerret (talk) 13:28, 19 October 2014 (UTC)

I'm going to say this complex and nuanced statement out loud because I believe that I can trust you all to be wise about applying it: If there's truly no problem at all with the information, then there's no need for a source. Wikipedia is an encyclopedia-writing project, not a fancy-sounding-footnote-producing project. Things like MEDRS exist to improve content, not to show off our ability to find gold-plated academic sources. If zero (actually zero) editors choose to WP:CHALLENGE the contents of that statement, then you don't actually need an ideal source, or often any source at all, even for biomedical information. WhatamIdoing (talk) 15:50, 19 October 2014 (UTC)

Electronic Cigarette is a hotly contested page. The activity on it is a lot. I should have said I dont have a problem with the material being there if it can be referenced correctly. It also goes to where its at in the article, placed in the lede at the time with weight questions. QuackGuru insisted it be in the lede. AlbinoFerret (talk) 17:17, 19 October 2014 (UTC)

This is one of those questions that's going to have to be answered with a resounding "it depends"; if you're unsure about the use of a source, discuss it on the article talk page, or open a thread at WP:RSN. The organization, the particular page, and the specific claim will all bear on question of reliability. TenOfAllTrades(talk) 03:46, 14 October 2014 (UTC)

MEDRS says major - that is a big caveat to any blanket OK. Jytdog (talk) 03:48, 14 October 2014 (UTC)

To be fair, there is never going to be a blanket "yes" or "no" to questions of reliability. Reliability always depends on the specific source and the specific text that it is supposed to be supporting. However, just because a specific page doesn't name an author or have specific cites does not automatically disqualify them as reliable. Yobol (talk) 03:51, 14 October 2014 (UTC)

I agree that the absence of specific, named authors or the absence of specific footnotes (or other citations) does not automatically render a source unreliable. I would also be inclined to presume that most pages provided by a small number of highly-reputable organizations (like the CDC) are probably trustworthy (but even there we will find exceptions). MEDRS has always required editors to engage their good judgement - and often demands more subject-specific knowledge than many other Wikipedia policies and guidelines - to be applied correctly and effectively. TenOfAllTrades(talk) 04:03, 14 October 2014 (UTC)

Naming the authors is never a requirement; see the next to last item in the Wikipedia talk:Identifying reliable sources/FAQ.

I tend to think of good websites as being lightweight sources of the sort intended by the adjacent statement, which is included in both WP:RS and MEDRS. They're perfectly fine for some purposes. They're best suited to lightweight claims, of the sort that any well-informed lay person would know (e.g., the common cold is a viral disease, antibiotics don't kill viruses, washing your hands can prevent some infections, etc.). When used for suitably lightweight claims, I'm happy to see them, because that gives a lay-accessible source for readers who want more information. WhatamIdoing (talk) 05:19, 14 October 2014 (UTC)

Hard to make a call on this without specifics. But I'd generally say that if there is a press release, report, FAQ or statement on the CDC, EMA, or FDA website, its because the organization backs it. And I can't think of any source that I consider more reliable than the CDC on the subject of infectious disease and/or epidemiology. If someone knows of a specific reason to doubt the reliability of a specific CDC page, I'd sure be interested to hear it. Formerly 98 (talk) 21:51, 16 October 2014 (UTC)

Centers for Disease Control and Prevention (CDC) (April 2014). "Notes from the field: calls to poison centers for exposures to electronic cigarettes--United States, September 2010-February 2014". MMWR Morb. Mortal. Wkly. Rep. 63 (13): 292-3. PMID 24699766. 

Centers for Disease Control and Prevention, (CDC) (6 September 2013). "Notes from the field: electronic cigarette use among middle and high school students - United States, 2011-2012". MMWR. Morbidity and mortality weekly report. 62 (35): 729-30. PMID 24005229. 

I'm sure you know the relevant part of MEDRS is Wikipedia:Identifying reliable sources (medicine)#Medical and scientific organizations.

User:Formerly 98, these sources are currently used at Electronic cigarette. QuackGuru (talk) 02:35, 18 October 2014 (UTC)

Why am I not surprised that QuackGuru forgot to link to the Electoronic Cigarette talk page discussions on these sources and the Project Medicine talk page where he also started a discussion on this very subject. He was answered a few times why the references he wants to use are not appropriate. Here is the page I started this about. AlbinoFerret (talk) 02:46, 18 October 2014 (UTC)

(edit conflict)And here is the reason (copied from myself on other talk) for MMWR being a primary source, and unusable to give medical information or contradict secondary review papers:

Let me quote the authors guide[16] for the Weekly reports that QG wants to use:

D. Notes from the Field

Notes from the Field are abbreviated reports intended to advise MMWR readers of ongoing or recent events of concern to the public health community, without waiting for development of a Full Report. Events of concern include epidemics/outbreaks, unusual disease clusters, poisonings, exposures to disease or disease agents (including environmental and toxic), and notable public health-related case reports. These reports may contain early unconfirmed information, preliminary results, hypotheses regarding risk factors and exposures, and other similarly incomplete information. No definitive conclusions need be presented in Notes from the Field.

That to me reads almost the very definition of a primary source in WP:MEDRS. If this can be used without caution, then i'm not sure why other sources are being rejected. --Kim D. Petersen 02:57, 18 October 2014 (UTC)

Gray literature

An editor, KimDabelsteinPetersen has claimed that a source fails WP:MEDASSESS and thus MEDRS for supporting health-related content, because it is Gray literature. The source is:

• 2014 report produced by the "Conference of the Parties to the WHO Framework Convention on Tobacco Control" at their Sixth session, Moscow, Russian Federation,13-18 October 2014 (which kim did here

Seems like an accurate categorization as "gray literature"; am interested in folks' thoughts on this and if we should include discussion of Gray literature in MEDASSESS or elsewhere in MEDRS. thanks! Jytdog (talk) 14:31, 2 November 2014 (UTC) (question has no meaning. striking Jytdog (talk) 22:32, 3 November 2014 (UTC))

Correction: I'm not saying that it "fails" WP:MEDASSESS - i'm saying that the WP:WEIGHT currently put on this particular example of Grey literature doesn't match the reality of the source. Claiming that it is authoritative, or a position statement or the like is simply not correct. This would (in my view) be a valuable resource, but it would be less useful than a fullblown WP:MEDRS review paper. Just like whitepapers, preliminary reports etc. all are useful, but with caveats. --Kim D. Petersen 15:01, 2 November 2014 (UTC)

fair enough. thoughts, then? Jytdog (talk) 15:13, 2 November 2014 (UTC)

I think MEDRS could benefit from a more indepth discussion on what constitutes policy statements, formal reports etc. and how to handle Grey literature in general. If the experience i can draw from Electronic cigarettes is a measure, then names such as WHO, FDA, CDC etc have an almost hypnotic effect on editors, so that they think that everything from these sources are more authoritative than the regular scientific literature. Some of the documents from these organizations are authoritative, but material in general from these isn't in that category. --Kim D. Petersen 16:14, 2 November 2014 (UTC)

I agree that currently MEDRS does not seem to clearly establish guidelines for use of literature from recognized organizations outside of formal reports or public position statements. The reliability of gray literature such as white papers, internal memos, conference proceedings, etc. needs some discussion, in particular as it compares to the peer-reviewed literature that meets current MEDRS guidelines. Mihaister (talk) 22:19, 3 November 2014 (UTC)

It says "Report by WHO". Yes WHO does not do a very good job getting their content out. But that is not a reason not to use it. Doc James (talk · contribs · email) 22:25, 3 November 2014 (UTC)

nobody would consider a conference abstract a secondary source. clearly the term "gray literature" is too broad to be usefully discussed. striking the query. done here. Jytdog (talk) 22:32, 3 November 2014 (UTC)

Impact factor of journals as the determining factor in weight

Assume we have two WP:MEDRS review papers, which are both written by respected experts/researchers within the topic of the publication and article where they should be used, and both journals fall within the topic area of the article in which they should be used:

A is published in a journal with impact factor 15 B is published in a more specialized journal with impact factor 5

Question(s):

• Should A be cited >3 times more than B? And why?
• Assume that A and B are both specialized journals would it now be the case?
• More specifically should impact factor be a determining factor in weight? Why? Why not?

--Kim D. Petersen 22:14, 4 November 2014 (UTC)

But you haven't told us what font each journal uses--how can we make a decision?

Seriously, there's a dilemma implied - either we ignore impact factors altogether, or we treat impact factors as the most important qualification - that isn't really useful. There's a whole bunch of unstated assumptions hiding in the background that will affect an assessment of reliability; it's never going to be a perfect apples-to-apples all-things-equal-but-impact-factor situation. Does "specialized" mean "specialized", or does "specialized" mean "fringe"? Are the two reviews reaching different conclusions, or is this about making sure every review writer gets a cite in Wikipedia? Are there multiple reviews on the topic, where the two reviews in question represent the most recent perspectives on a still-controversial issue? Is your first question about number of cites absurd? (I can answer that last one: yes.)

Is this part of some larger dispute, and not nearly as hypothetical as you seem to be framing it? Honestly, WP:RS and WP:MEDRS very much depend on context. TenOfAllTrades(talk) 22:29, 4 November 2014 (UTC)

@TenOfAllTrades: assume that both journals are MEDLINE indexed (Index medicus), and specialized means that the first is more generic within the topic, while the other is specialized on a subtopic areas, but both are on-topic for the article in which they are used. None of the journals fall within the fringe area (or even tiny minority one), as the impact factor should indicate. Both are respected journals with decades of publication. --Kim D. Petersen 22:39, 4 November 2014 (UTC)

As an additional question what if the journal has an impact factor of zero? Doc James (talk · contribs · email) 22:30, 4 November 2014 (UTC)

Impact factor 0 journals wouldn't be in MEDLINE would they? --Kim D. Petersen 22:40, 4 November 2014 (UTC)

Oh yes they would, MEDLINE has journals that have no IF. MEDLINE and the JCR are independent, although in practice they will, of course, often arrive at similar conclusions about the value of a certain journal. --Randykitty (talk) 11:18, 5 November 2014 (UTC)

As an additional additional question, what if the "high quality review" is being savaged by the people whose work it reviewed for misrepresenting and distorting their research? There are serious problems with the Grana review, and to cite it 3-4 times more often than any other is a major WP:WEIGHT violation, even if a small clique like what it says..--CheesyAppleFlake (talk) 22:35, 4 November 2014 (UTC)

• Some background. The article is electronic cigarettes. The review in question has a lead author who is a tobacco researcher at UCSF by the name of Ghana. It was published in the Circulation (journal) in May 2014 under the title of "E-cigarettes: a scientific review". Circulation has one of the highest impact factor or any journal in medicine.
• It conclusions are similar to those of WHO, the AMA, the ALA, the British Medical Association (BMA), etc. as mentioned here [17]
• The article has a POV tag because some do not like this review or the WHO, AMA, and BMA's positions.

• Some want to use this article from a journal with an impact factor of zero instead [18] Author is from DeBusk College of Osteopathic Medicine. UCSF I know as it is consistently ranked as one of the top universities in the world for research but never heard of DeBusk. Doc James (talk · contribs · email) 22:43, 4 November 2014 (UTC)Doc James (talk · contribs · email) 22:43, 4 November 2014 (UTC)

I would point out that this is not an accurate description of the problem i'm presenting. I was hoping to get a more generic answer. --Kim D. Petersen 22:46, 4 November 2014 (UTC)

Gotta give it to Doc James for the painfully "neutral" and "accurate" portrayal of the debate. Mihaister (talk) 23:08, 4 November 2014 (UTC)

By way of disclosure I am involved in the debate that forms the motiviation for this question to be posted. But in my experience as someone who has been involved in research my whole career, the impact factor of the journal says a lot about how the author is percieved by their peers. Everyone in research tries to get their papers into the most prestigious journal possible (why wouldn't you?). But the NEJM or other high prestige journal is only going to publish one or two reviews on e-cigarettes this decade. You can bet that the guy who published in the lower impact factgor journal tried to publish in the higher impact factor journal but couldn't. I've published dozens of papers in the Journal of Organic Chemistry, but can't get my papers into Nature, Science, or PNAS. This isn't really even a debateable topic for those who have made research their career. It's a no-brainer. Formerly 98 (talk) 22:47, 4 November 2014 (UTC)

• Impact factor matters, but for Wikipedia purposes, I'd really only be using it to determine if we're dealing with a legitimate or very new journal where there's either a questionable reputation or none at all. Basically this question is getting towards this last paragraph [19]. Essentially, we want to guard against articles where someone tried to publish an idea that's not accepted and just went for a journal that would publish practically anything. A low impact factor may be an indication of that, but it's a tricky question for us to address here (but still something to be mindful of). Now if we had a scenario like Kim D. Petersen presented above if we had already determined the journals otherwise met our reliability standards, we should not be using impact factor to them determine the weight of those sources for a myriad of reasons those of us who actively publish are probably familiar with that we probably don't want to involve ourselves in as Wikipedia editors. However, reading over Talk:Electronic cigarette, it seems there's a lot more going on than what's being presented here. It's looking like there might be some underlying POV issues or else a very problematic misunderstanding of scientific research that's making that talk page quite a bearcat to follow (Yobol had a good summary [20]), so just a heads up for anyone venturing over there to dig into the background. Kingofaces43 (talk) 23:32, 4 November 2014 (UTC)

  I would agree with the above, including (to an extent) with Yobol, in that particular instance. And yes, there are underlying POV issues (see this) :) --Kim D. Petersen 23:39, 4 November 2014 (UTC)

Kim, I'll just say that looking over the talk page, your hands don't look particularly clean in that regard either, and attempts to point that out have resulted in deflections on your part towards other editors. As an uninvolved editor, seeing you open with a hypothetical question above followed by the "cat out of the bag" response sets off a WP:TE red flag for me. This isn't the place to discuss editor behavior in general, so I'm just pointing this all out in the hopes that you slow down for a bit to really heed what other folks are saying a bit more in regards to medical topics to hopefully reach a resolution quicker. Either way, I plan to continue resume my 10 foot pole approach on the article (didn't realize it was being commented on until halfway through my first reply), so good luck. Kingofaces43 (talk) 01:31, 5 November 2014 (UTC)

The problem here is that this is not a medical topic. It's an article about a consumer product. Unfortunately a group of editors are insisting it is a medical topic so they can use MEDRS to exclude any sources that don't fit their POV.--CheesyAppleFlake (talk) 06:02, 5 November 2014 (UTC)

It doesn't matter whether it's a medical topic" or a "consumer product topic". What matters is whether the specific bit of material is biomedical or not. (NB that socio-medical material is not covered by MEDRS, a fact that will surprise some of its most enthusiastic supporters.) This means that some sentences in that article need MEDRS-compliant sources, and others don't. "Devices are manufactured in China" and "The advertising is in this style" are not biomedical sentences; use any RS (including, if you want, a MEDRS-style source). "The dose of nicotine delivered is poorly standardized" is biomedical, so you need MEDRS sources for that. It's not actually difficult to tell which sentences need which kind of sources, but you do have to go through the article sentence by sentence to find out. WhatamIdoing (talk) 15:20, 5 November 2014 (UTC)

@Kingofaces43: fair enough. I deliberately do not delete anything from my talk-page, no matter what it is. I never claimed to have "clean hands" - but the reason that i did the "Cat out of the bag" is because other two other editors at that point had included content issues directly, that weren't particularly neutral or correct descriptions of the issue. I called it a "cat out of the bag" as a bit of a joke, sorry if it raises a red flag for you. I truly attempted to describe a specific issue in as neutral and generalized way as i could, without having this turn into a content debacle. --Kim D. Petersen 02:14, 5 November 2014 (UTC)

Cat out of bag

Since the Cat is out of the box: The two journals are: Circulation[21] (impact factor ~15) and Addiction[22] (impact factor ~5) and the papers are [23][24]. According to Doc James, and other editors, the Circulation paper must naturally be cited/used more (according to impact factor) in the article than the Addiction one, because it is in a "more respected journal", while the other side is that these are roughly equal in reliability, because both are in respected top tier journals. --Kim D. Petersen 22:56, 4 November 2014 (UTC)

And both those papers are cited multiple times. Doc James (talk · contribs · email) 23:10, 4 November 2014 (UTC)

Correct. The controversy originates from the fact that the Circulation paper is given a rather extreme amount of WP:WEIGHT: Cited in the article no less than 45 times, more than 4 times as much as the Addiction one (which as far as i can tell is the second most quoted review there). And the argument given when people point this out is that the impact factor merits it. Thus the original question. --Kim D. Petersen 23:25, 4 November 2014 (UTC)

We also have to acknowledge that the paper in Circulation is not about cardiovascular medical topics, but e-cigarettes and the evils of addiction to tobacco products. Per WP:MEDSCI "Be careful of material published in a journal ....... or that reports material in a different field." AlbinoFerret (talk) 03:32, 5 November 2014 (UTC)

Most of what this review supports could be supported by many other high quality sources. Here are some of the text bits it supports:

• "E-cigarette devices are predominately manufactured in China"
• "and devices are often manufactured in China."
• "E-cigarette brands have increased advertising with similar marketing to that used to sell cigarettes in the 1950s and 1960s."

Doc James (talk · contribs · email) 23:37, 4 November 2014 (UTC)

This is just another WP:IDHT response. The impact factor is just one variable of many, and the idea that the ratio of the number of cites needs to match the ratio of the impact factors is ridiculous. If the lower-impact factor journal's position was wildly out of sync with what most other sources say, depending on other factors it's possible it shouldn't be cited at all regardless of impact factors. Look to what other authoritative sources say. For example, what does the World Health Organization say? Zad68 01:52, 5 November 2014 (UTC)

Zad68, it is not so ridiculous that User:Doc James didn't (at the very least) imply this today[25] - i don't know who is not hearing things, but this certainly can be read as such a statement. Combined that with other comments of the same kind.... And i tried to limit this specifically to the impact factor issue, so i'm not going to comment on anything else. --Kim D. Petersen 02:06, 5 November 2014 (UTC)

In the situation above, since we're only considering IFs, I think there is a line to be drawn, such that journals w/impact factors above this point are OK to be used but below that point would not be OK to be used as per MEDRS. Maybe the number of times other researchers have cited a paper, or whether it has been endorsed by major medical bodies, should be used to determine its reliability rather than impact factor. Jinkinson talk to me 02:23, 5 November 2014 (UTC)

It also depends on what other sources exist. Doc James (talk · contribs · email) 02:52, 5 November 2014 (UTC)

I have no stake in this specific contest, and have academic but not medical expertise. Nevertheless I agree with what seems to be the general consensus above: a very low impact factor can be used as a sign that a journal might not be a reliable source, but once we're agreed that two sources both are in legitimate and respectable scientific journals (as appears to be the case here) then the impact factor tells us nothing about how much weight to assign these sources and should not be factored into these debates. It is a very indirect measure (of how much other papers in the same journal have been cited within a narrow time window) of things that can be judged better by more direct measures (e.g. how much the papers themselves were cited, and what the citations actually say about them). So using it instead of these other better measures only adds heat to the debate without adding light. --David Eppstein (talk) 03:45, 5 November 2014 (UTC)

David's response is perfectly reasonable. Mine is perhaps a little more extremist: I would recommend against using impact factor at all, and especially not during a dispute. You can use it, if you'd like, to choose between the sources you want to add yourself. But when evaluating a source that some other editor added, I recommend focusing on actual substance, like whether the author knows anything about the subject and whether the journal is likely to have knowledgeable reviewers. And, as David says, once you've established that a journal has a reasonable impact factor for its subject, then you should stop thinking about impact factors. You might use a truly low impact factor to rule out a paper that you already thought was dubious, but you should never use it as a means of choosing one paper from an obviously decent journal over another paper from another obviously decent journal. (I wonder if we can get WP:TBANs against discussing the impact factor of sources for people who persist in thinking that this is hugely important?)

I'd also encourage people not to count the number of citations (as if that means anything: if I repeat the same citation after every noun in the same sentence, am I really "giving it more weight"? If I spam it after every general/background sentence--sentences that it happens to be capable of verifying, but sentences that were present long before the paper was even a gleam in the author's eye--am I really "giving it more weight"?).

If most (or all) sources are saying the same thing, then you are not giving any of them "undue weight" by adding any of the citations behind those sentences. Undue weight is when your sentences change because you are paying too much attention to an abnormal source. WhatamIdoing (talk) 06:06, 5 November 2014 (UTC)

• Given that this discussion is about the IF and its possible role, I have not looked at the sources/dispute that this question originated from. Much can be said about the IF and many people hate it. Personally, I don't like it much (it grates against my statistical side that it's a mean of a highly non-normal distribution, without any standard error or something similar), but recognize its (limited) utility and do hate the abuse of the IF in evaluating research or a researcher. In the life sciences/medicine, I consider anything that has an IF above 2 as most likely a respectable source, although there are exceptions. Below 2, many journals still publish lots of good articles reporting on important research (just not "sexy" and flashy enough for the higher impact ones). High impact journals do not always publish the best science: it's well known that such journals have also the highest rates of retractions (although one may debate whether that is because they are not selective enough, people are so desperate to further their careers that they will "teak" their data to get into a high IF journal, articles in these journals are exposed to more scrutiny, or a combination of all that). If the question at the top of this section would be about an article in a journal with an IF of 15 (or 5, or 2) as opposed to one without an IF (but still in MEDLINE), I'd probably give more weight to the one in the journal with an IF. The difference between 5 and 15 is much smaller, I think. A few years ago a colleague of mine published an article in PNAS (IF around 10) and almost simultaneously I published one in a journal with an IF around 3. When our institute was going to be evaluated, he reproached me for not trying hard enough and not publishing in high IF journals. Yet, my article was cited 3 times as often as his. Despite the IF difference, my article certainly had more impact than hisd. Was my work better/more important? Well, that is of course a completely different question and to answer that one, one actually needs to look at the articles themselves and read what those different citations actually said about both articles (perhaps mine got cited more because many people were commenting on how bad it was...). The short of this long comment is that, as David and WhatamIdoing say above, it all depends. IF is often used as a shortcut and, like all shortcuts, that has its risks. --Randykitty (talk) 11:39, 5 November 2014 (UTC)

The best source

About this:

Yes, you are allowed to edit this guideline!

I removed a statement that we should cite (only) "the best" sources because it isn't true. We require the use of sources that meet the minimum requirements. We do not require "the best" source.

In addition to not being true, it is unworkable. I'm convinced that we frequently will not be able to agree on which source is the best. Is the best source an decent journal or a good textbook? Is the best source the one published in a good, open-access journal last month, or the one published yesterday in a paywalled journal? The one that is written in English by someone nobody's heard of, or the one written in French by the foremost authority? The one that makes us look like we're super-smart, or the one that 80% of our readers will be capable of reading and understanding? Is an obituary written by a friend in an academic journal better than a thoroughly fact-checked obituary in a newspaper?

If everyone agreed on "the best source", and you cited everything to that one "best" source, then why did your article get slapped with a cleanup template that complains about your sourcing?

What if "the best" source is actually pretty lousy on an objective scale?

I think we are better off recommending all "high-quality" sources (and only high-quality ones) rather than "the best". WhatamIdoing (talk) 15:58, 5 November 2014 (UTC)

Yes, a very complicated issue. I've not been a huge fan of Evidence Based Medicine in the past, at least to the extent that many others are, but dueling "expert opinions" over at the e-cigs article has given me pause with respect to expert opinion. We have the full gamet over there, from arguments (I have participated in) about the relative merits of expert opinion from journals that both have high impact factors (one higher than the other) to those expressed in a review written by an author with no prior publications in the field. If I, as a synthetic organic chemist, write a review on e-cigarettes, schizophrenia, or crop rotation and find someone willing to publish it, can I be cited as expert opinion in Wikipedia? Will someone use Wikipedia's voice in parroting my opinions? How much simpler things are if we restrict ourselves to the facts!

So while hearing what you say, my POV is that we have to discriminate between the quality of sources of expert opinion in some way, though I'm not quite clear how one would do so. Many if not most of our pharmacology articles caveat the conclusions of industry funded studies, implying that this COI makes their conclusions less reliable. I've even seen a meta analysis discounted by the statement that the author once did consulting work for a contractor of the drug manufacturer. But its ok to quote as expert opinion the conclusions of someone with no little or no prior publications in the field, no clincial trial experience, maybe even no experience working with patients without noting these items? I see a problem here, I just don't know if there is a solution that is not worse than the problem. Formerly 98 (talk) 16:56, 5 November 2014 (UTC)

On a humorous and somewhat related side note, my edit to MEDRS has just been reverted by an editor with little or no prior history of editing medical articles. So much for value of expertise in the 21st century! :>) Formerly 98 (talk) 17:02, 5 November 2014 (UTC)

That gave me a good laugh! True, I don't edit medical articles very much, on Wikipedia. In real life though... Be careful what you say about an anonymous editor. On the internet you never know who's at the other side. I could be a chimp hacking away at a keyboard and occasionally making sense just by chance. Or I could be a researcher who sits on a dozen editorial boards and is an editor of some other journals... And when we talk about expertise, for all I know, you're one of the students down the hallway. Not that all of this matters, we're all equal here. --Randykitty (talk) 17:34, 5 November 2014 (UTC)

Nothing personal intended. It was just humorous that the revert was so closely related to the topic of discussion. Your input is of course completely welcome.

And just to quibble a little bit, even if the sources available are all "bad", "best" is a relative term as you noted, and one would want to use the best available irrespective of its quality on an absolute scale, no? And of course we have minimum standards below which one would not write an article at all. Formerly 98 (talk) 17:43, 5 November 2014 (UTC)

Exactly. And it is the latter point that gets lost in your proposed wording. --Randykitty (talk) 17:54, 5 November 2014 (UTC)

Actually its not my proposed wording. Its the wording that has been there since time immemorial and prior to WhatamIdoing's edit. I wasn't proposing new language so much as expressing the opinion that the revision was worse than what was already there. Why don't you propose something that addresses both problems? (It would be nice if the regualar medical editors were nearly as interested in this issue as you are) Formerly 98 (talk) 18:11, 5 November 2014 (UTC)

In fact, I think the current text is pretty good. "A Wikipedia article should cite high-quality reliable sources". That implies that if no high-quality RS are available, we cannot write an article. It's reasonably objective: more people will agree on what constitutes a "high-quality reliable source" than what is the best source. --Randykitty (talk) 18:39, 5 November 2014 (UTC)

I wasn't implying use of a single source as suggested by the language you used. Certainly the italicized "the" has nothing to do with anything I've posted here, and is a misunderstanding of my position. I've stated only that one should use the best sourceS available. If there are 60 articles meeting the absolute miniumum requirements of MEDRS, you don't use a dart board to select which ones you use. You use the standards described in WP:MEDASSESS, aspects of WP:MEDDATE and the paragraph just below it on Independent Sources, the entire section entitled "Biomedical Journals", "Medical and Scientific Organizations", and "Other Sources". In my opinion it is unrealistic to suggest that we are not already engaged in discriminating "fair", "good" and "excellent" sources from one another every time we write or edit a biomedical article.

That being said, experience shows that after this many iterations, we are not likely to come to agreement. Lets see what others say (if they comment at all). Thanks for your input. Formerly 98 (talk) 18:57, 5 November 2014 (UTC)

+----------------------------------------------------------------------------------------------------+ I see what you mean, I misunderstood partially. No, I indeed disagree with that. Citing all good sources is overkill and unencyclopedic, IMHO. --Randykitty (talk) 19:32, 5 November 2014 (UTC)

"Since time immemorial" means since 2008, when that sentence was in a section titled ==Accessing sources==. The edit summary indicates the existence of discussions that I haven't pulled out of the archives yet. I would be surprised to find that they said that high-quality reliable sources that weren't "the best" should never be used. Instead, I'm betting (from the context of the edit) that the discussions will be all about "WP:PAYWALL is policy, and you should use great sources even if they're not freely accessible". WhatamIdoing (talk) 20:18, 5 November 2014 (UTC)

Fair enough, though the merits of the different approaches are probably more important than the history. In any case, as no one but yourself, Randykitty, and I have taken an interest in this, it looks like the edit will stand. Formerly 98 (talk) 22:52, 5 November 2014 (UTC)

I actually liked that the term best was in there, but I never read it as saying we should be trying to figure out exactly what the "best" source is. Instead it read as that we should be reaching for the best sources we can (i.e., not using borderline gray literature when we have plenty peer-reviewed reviews in journals to rely on in a given topic). There's a difference between saying we should be using a particular kind of source and establishing the mentality we should be using when looking at how we should use a source. I like the term best to help describe the spirit of the guideline, but the term can be tricky if we're talking about a concrete how-to for determining reliability. I'd like to see best used initially to orient folks towards the kind of thing we're looking or in general, then dive into what actually constitutes reliability, etc. Am I making sense, or is that getting a little too abstract? Kingofaces43 (talk) 01:10, 6 November 2014 (UTC)

"Best" vs. "reliable" sources in the MEDRS guideline

This is quite an interesting issue raised by @WhatamIdoing: in a proposed edit to MEDRS. I'd say that as long as we are reporting facts, "reliable" is sufficient. The problem is that almost every secondary source contains a mixture of data and interpretation/opinion, and in these cases we probably want to be careful about cherry-picking sources based on the opinions expressed. I've raised this issue in the past with respect to the Cochrane reviews, which typically include a mixture of conclusions based on relatively inarguable statistical analysis of clinical trial results and others that are largely just the opinion of the authors. Our article on Aripiprazole for example, states that this widely used antipsychotic "may" be useful for treating schizophrenia. The referenced Cochrane review found that it reduces relapses, but expresses the viewpoint that not enough endpoints have been measured to truly determine whether the drug helps people have a better life. This latter statement arises not from statistical analysis of the trial results, but from the intuition and judgment of the two co-authors, neither of whom appear to have ever designed a clinical trial and who have 21 prior publications in total between the two of them. So two questions: Should reliable sources suffice for facts and "best" sources be required for opinion? How would we recognize "best" sources for opinion? In particular in the case of aripiprazole, this can be more narrowly phrased: Does the production of a state of the art statistical analysis render opinions not mathematically derived from that statistical analysis authoritative, such that these opinions should be offered here in Wikipedia's voice? I'd be interested in people's thoughts on this. Sorry if this rambled. I think it is an interesting issue. Formerly 98 (talk) 16:05, 5 November 2014 (UTC)

I think such judgment would require us to assume Wikipedia editors are experts in the subject and are able to accurately and neutrally discriminate from within the conclusions of reliable sources between statements based on "statistical analysis of clinical trial results" and statements that represent "the opinion of the authors." Furthermore, even if actual subject-matter expertise is not necessary to distinguish "analysis" from "opinion", it would still constitute WP:OR. I think we must be careful here and avoid fanning the flames of the old debates on experts editors and credentials. Mihaister (talk) 19:43, 5 November 2014 (UTC)

In general, with regard to relatively new drugs used in psychiatry, I would guess that the views of published and peer-reviewed opinion leaders would be significant, and would be relevant to include alongside alternative statistical analysis. I do not know what to say about defining or identifying a good source for opinion. I think that reliable sources for facts can be easier to identify, but not necessarily by everyone. A small crowd may not have much wisdom, so in my opinion, the best that conscientious editors can do ask for assistance at WikiProjects and hope that a bigger crowd has a bit more wisdom. Snowman (talk) 22:10, 11 November 2014 (UTC)

Isn't "opinion leader" the pharma jargon for "physician who prescribes a lot of drugs, and we hope that if we flatter him and give him special training, that he'll prescribe more of ours and less of our competitor's drugs?" I'm not sure that we want to follow the views of "opinion leaders". WhatamIdoing (talk) 04:23, 12 November 2014 (UTC)

I assume that Snowmanradio intended - by his explicit use of the adjectives "published and peer-reviewed" - to refer to scientists and clinicians who are widely published (and cited) in the peer-reviewed literature, rather than to a winking bit of marketing jargon. But hey, why waste a perfectly good opportunity for a bit more unhelpful snark? TenOfAllTrades(talk) 05:35, 12 November 2014 (UTC)

Other sources

The "Other sources" section on Wikipedia:Identifying reliable sources (medicine) says; "Peer-reviewed medical information resources such as WebMD, ...". As an example, lets consider; Women's Health on webmd.com, which is used a as source in the Vagina anatomy article. I have been trying to decide if this webpage is compliant as a medical source or not for the Wiki. Near the bottom of the source webpage there is a paragraph under the heading "From our sponsor" that says that the webmd team have not reviewed the content for accuracy. Hence, I do not see any evidence that the main medical content of this webpage is peer reviewed and the evidence that I do see appears to say that is is not peer reviewed. Any comments? Snowman (talk) 13:28, 3 November 2014 (UTC)

It appears that the "From our sponsor" is a description of that particular section, specifically "Osteoarthritis Knee Pain". If you click on any of the three links in that section, the three pages each have a disclaimer that the editorial content comes from a sponsor. Basically the knee links are from the sponsor, not the Women's Health page, so it appears that the Women's Health page appears to be still under the editorial control of WebMD. Yobol (talk) 15:07, 3 November 2014 (UTC)

I see. Thank you for explaining that to me. Looking at it differently, the Web MD editorial policy says that the Web MD authors write their pages from reliable sources, but it seems to say that all the reviewing is done in house. I would have thought that a key feature to peer reviewing is that the reviewers are external or independent of the authors. Snowman (talk) 21:12, 3 November 2014 (UTC)

WebMD is not a good source generally. Doc James (talk · contribs · email) 22:27, 3 November 2014 (UTC)

The trouble is that I have seen people quoting the "other sources" section to justify using the WebMD as a source and only quoting a portion of the section that would appear to be favorable to the website. Perhaps, the emphasis of the "other sources" section could be amended to have a greater tenancy to discourage people from using it as a source. Does "reviewed in house" have the same meaning as "peer reviewed"? Snowman (talk) 23:37, 3 November 2014 (UTC)

Snowmanradio is referring to me by stating that "I have seen people quoting the 'other sources' section to justify using the WebMD"; however, I told him at the Vagina article talk page that I have been planning on removing that source or using it as an adjunct (as a laysource) to scholarly sources, and did use it as an adjunct already. That is allowed by WP:MEDRS. In fact, there are currently two WebMD sources, and a Mayo Clinic source, in Clinical significance section of the Vagina article. And I have been planning on removing all of them and/or using them as adjuncts to scholarly sources. Indeed, the Other sources section of WP:MEDRS currently states, "Peer-reviewed medical information resources such as WebMD, UpToDate, Mayo Clinic, and eMedicine are usually acceptable sources in themselves, and can be useful guides about the relevant medical literature and how much weight to give different sources; however, as much as possible Wikipedia articles should cite the more established literature directly." Because of what that section currently states, and the general reliability of WebMD, I don't see any valid reason to "have a greater tenancy to discourage people from using [WebMD] as a source"; and if we are going to do that, the same should apply to the other health sites. However, the use of these sources has already been discussed at this talk page time and time again. See, for example, this 2011 discussion about eMedicine and Merck Manuals and this 2013 discussion partly commenting on WebMD; WP:Med editors have always had differences of opinion regarding these sources. Flyer22 (talk) 10:58, 4 November 2014 (UTC)

To me, it does not seem to be efficient to write in-line references for WebMD sources and then remove them. Perhaps, the "Other sources" section could have a greater emphasis against using certain sources. User:Flyer22 said that; "And WebMD passes, and is listed in, the Other sources section of WP:MEDRS as an acceptable medical source to use." on the Vagina talk page in this edit. Note the emboldened text seen in User Flyer22 edit. Snowman (talk) 13:40, 4 November 2014 (UTC)

And who stated that I added all those sources? I certainly did not state that. Nor did I imply that. If you want to know how all those sources got into the article, the article's edit history is at your disposal. But do I sometimes use WebMD sources on Wikipedia? I indeed do, usually as an interim for scholarly sources and/or as an adjunct to scholarly sources. Using a laysource in combination with a scholarly source has repeatedly proven beneficial for readers at various medical articles, or specifically at anatomy articles, especially in a WP:PAYWALL case. That's also why Template:Citation Style documentation/lay exists. WP:Reliable sources that accurately explain medical matters in plain English are always beneficial. And this is why WP:MEDRS sees WebMD, UpToDate, Mayo Clinic, eMedicine, and similar generally reliable medical sources, as "usually acceptable sources in themselves" that "can be useful guides about the relevant medical literature and how much weight to give different sources." That is why I stated "WebMD passes, and is listed in, the Other sources section of WP:MEDRS as an acceptable medical source to use." If you have not read the arguments for and against using WebMD, and similar sources, in the aforementioned 2013 discussion I linked above, then I suggest that you go ahead and do that. And since this page is on my WP:Watchlist, there is no need to ping me to it via WP:Echo. Flyer22 (talk) 00:16, 5 November 2014 (UTC)

I found an edit adding some in-line reference points to WebMD in the vagina Wiki article that User:Flyer22 added back on 27 April 2014 with this edit. Snowman (talk) 21:15, 11 November 2014 (UTC)

Snowmanradio, again, "And who stated that I added all those sources? I certainly did not state that. Nor did I imply that." As the diff-link you added shows, the "Vaginal Problems -- Home Treatment" WebMD source was already in the Vagina article. The only WebMD source I added to that section is the aforementioned "The Vagina (Human Anatomy)" WebMD source. And so what that I did? That source is not terrible or even poor, as far as I can see, and I've already been very clear above that "I indeed do [sometimes use WebMD sources on Wikipedia], usually as an interim for scholarly sources and/or as an adjunct to scholarly sources. Using a laysource in combination with a scholarly source has repeatedly proven beneficial for readers at various medical articles, or specifically at anatomy articles, especially in a WP:PAYWALL case. That's also why Template:Citation Style documentation/lay exists. WP:Reliable sources that accurately explain medical matters in plain English are always beneficial." As can also be seen in that diff-link, this Mayo Clinic source was already in the Vagina article; I did not add that source, but I also did not see a need to remove it, especially since, as that diff-link shows, my intention was to fix up that section and expand it, not make it perfect. Building a Wikipedia article is a step-by-step process. And, in my experience, those concerned with getting an article immediately perfect usually are not good article builders. I also reiterate that there is no need to ping me to a talk page that I am clearly watching, unless you are trying to get people to click on my username. I clearly stated above that there is no need to ping me to this talk page. And yet you did it anyway. Also, if someone wants to click on my username, it clearly was already linked above. The only reason that I linked your username with this post is because you linked my username despite my obvious request not to, and because I want to make sure you are aware of this post. Flyer22 (talk) 07:13, 12 November 2014 (UTC)

This is one of those frustrating perennial discussions. Is WebMD a good source? Nobody knows. Anyone who claims to know is being sloppy. Why? Because you simply can't judge whether anything is a "good source" until you consider how you want to use it. WebMD is a perfectly fine source for for lightweight claims (Is the vagina part of female anatomy? Do females usually have just one? Do non-human female mammals have one?) It's too weak for controversial subjects (Will 24% of women with vaginal cancer die from it? Does this drug have this side effect in 72% of women who take it?).

Personally, I think it's fine to use a source like that for a couple of lightweight, almost-sky-is-WP:BLUE claims, because giving decent lay-oriented sources is a valuable service for younger and less-technical readers. But the question you asked is unanswerable: it might be fine, and it might not, and it depends significantly on the sentence you plan to put the website after. WhatamIdoing (talk) 04:28, 5 November 2014 (UTC)

That is a good question to ask; reliable as a source for what? I think that the main page could be clearer about using WebMD and the main page may be slightly misleading saying that WebMD is peer reviewed, because saying that generally means peer reviewed by external or independent reviewers, as far as I am aware. Snowman (talk) 21:15, 11 November 2014 (UTC)

I think that Yobol was clear above about the editorial process of WebMD. I also think that I and WhatamIdoing have been clear about the usability of WebMD. You are never going to get WP:Med editors to be fully in agreement on its use; same for the other health sites in the Other sources section that you seemingly don't have an issue with. Flyer22 (talk) 07:13, 12 November 2014 (UTC)

Six-year study, reported at the annual meeting of the American Heart Association

Copied here from Wikipedia:Reliable_sources/Noticeboard#Six-year_study.2C_reported_at_the_annual_meeting_of_the_American_Heart_Association User:Fred Bauder Talk 11:21, 18 November 2014 (UTC) One source was "Study Finds Alternative to Anti-Cholesterol Drug" in The New York Times, but nearly all news sources had some version of this information, and often commentary by their medical reporter on its significance. "Study shows cholesterol drugs reduced cardiovascular risks" is the blog entry on the American Heart Association website.

Ezetimibe (edit | talk | history | protect | delete | links | watch | logs | views)

All mention of this finding, as well as the controversial history of the testing of this drug has been scrubbed from the article based on Wikipedia:MEDRS. User:Fred Bauder Talk 09:30, 18 November 2014 (UTC)

As health-related content, this is covered by WP:MEDRS which generally rules out the use of primary sources of any kind, as well as popular media. If you think this is a special case, the question should really be posted at WT:MEDRS Formerly 98 (talk) 10:34, 18 November 2014 (UTC)

Agree. We would need to find mention of this study in secondary sources. If it was significant, that'll be easy. Alexbrn ^{talk|contribs|COI} 12:00, 18 November 2014 (UTC)

Fred, do you want to withdraw your posting at RSN or do you really want to WP:FORUMSHOP this? And please WP:AGF and you know, talk to people like they are humans? Why don't you ask me why I cleaned up the article? Jytdog (talk) 13:08, 18 November 2014 (UTC) (strike, as it has been redirected by an admin Jytdog (talk) 13:54, 18 November 2014 (UTC))

folks who watch this board, here is the series of edits i made, to bring the article into line with MEDMOS and MEDRS. Doc James swung through and did some polishing. i had previously posted a question about what to do with the controversy section here btw. Jytdog (talk) 13:27, 18 November 2014 (UTC)

i made a note of at Project Medicine of the three two ongoing discussions about the new trial results. Jytdog (talk) 13:33, 18 November 2014 (UTC) (change 3 to 2 Jytdog (talk) 13:53, 18 November 2014 (UTC))

• I collapsed the RSN discussion and left a link from there to here. Preliminary individual study results shared at a conference before publication and touted in the popular press are exactly the kind of thing WP:MEDRS is set up to guard against. If the intervention works and is significant, before long we'll have excellent-quality secondary sources that will be appropriate to use in our articles. There's no reason to rush to try to get this in. Zad68 13:40, 18 November 2014 (UTC)

thanks for collapsing, zad! Jytdog (talk) 13:48, 18 November 2014 (UTC)

I have to admit I feel torn about this. Being consistent in following policy certainly avoids a lot of problems. But it really seems odd not to have any mention at all that this trial has reported out in the article, and I believe that WP:MEDRS permits primary sources in the case in which there is consensus on the Talk page. We avoid the use of primary research sources in part to avoid cherry picking, but cherry picking really is hard to do when you are talking about 9 year, 18,000 patient CV outcomes trials. There just arent' that many of them. Would it be completely out of line to include a single sentence stating that the trial has reported out and that analysis is ongoing? (I know I'm going to get slaughtered for this suggestion, but I had to throw it out there anyway. :>) Formerly 98 (talk) 14:01, 18 November 2014 (UTC)

• No need to feel afraid! WP:MEDMOS allows for a Research section and I think the study quality and sourcing available probably rise to the level of such a mention there. Zad68 14:09, 18 November 2014 (UTC)

(edit conflict)  :) not slaughtered! i am pretty firm that WP is WP:NOTNEWS. i know twitter is all aflame with this but the question I ask, is what can we actually say healthwise, that is meaningful? i would say "nothing" at this point. WP:MEDPRI gives room to report Big Clinical Trial Results by attributing them to the source, until they are discussed in a secondary source, after which we can remove attribution and report in WP's voice. but we don't even have a real WP:PRIMARY yet - just a conference abstract/presentation. Zad, I think that kind of thing is pretty temporal and newsy... it may tamp down people being all unhappy the News is not even mentioned, but people are inevitably going to push to include what the announced results are, under the same reasoning. I just don't think we should go down that path. Jytdog (talk) 14:11, 18 November 2014 (UTC)

I would say the following:

• We have no idea what the opinion of experts will be regarding the meaning of these results, their applicability to wider patient populations, or their clinical significance.
• But the result itself, that in a particular patient population, CV events were modestly reduced and there was no effect on mortality, is extremely unlikely to change. The only way that will happen is if the sponsor is lying. This is unlikely in part because the patent expires in 1.5 years, so why bother. And given the lack of indepedent access to the data by third parties, the publication of secondary sources will not expose any lies anyway, so they don't change this.
• I propose that we give a very narrow statement about the claimed results of the trial, attribute it to "according to Merck", and state that the clinical ramifications of the trial results are still under debate. Formerly 98 (talk) 14:24, 18 November 2014 (UTC)

• In my view I would not go this far. I would not actually try to bring content into the article that pulls the results into the content, not yet, not without MUCH better sourcing. Zad68 14:28, 18 November 2014 (UTC)

• (e/c) ...and I thought I was a hard-liner!  :) Taking a closer look at the sourcing, I see this is a borderline case, one where in my judgment the case for a Research mention could start to be made. To be clear, I mean no more mention than something like "Ezetimibe was the subject of a large study investigating its safety, ability control LDL cholesterol levels, and potential to improve cardiovascular health." At first glance the sourcing links looked significant enough, Forbes and NPR, but now I'm seeing there's really not that much in the authoritative sources and even those are specialist news-bloggers. It's really on the edge, in my opinion. Zad68 14:26, 18 November 2014 (UTC)

it made the NY Times too for what that is worth. i will yield to consensus, but my !vote remain a firm "not yet".  :) am interested to see what other folks say as the day unfolds. Jytdog (talk) 14:33, 18 November 2014 (UTC)

I don't feel real strongly about this, just my usual habit of thinking out loud. Formerly 98 (talk) 14:47, 18 November 2014 (UTC)

OK, i just yielded: here is the dif and then moved the new paragraph to the bottom. Jytdog (talk) 14:49, 18 November 2014 (UTC)

The NYTs is not a good source for medical content. We currently however are not using it for medical content just the reporting that a trial has been done. The one thing that stands out is "eventually a difference emerged. There were 6.4 percent fewer cardiac events -- heart disease deaths, heart attacks, strokes, bypass surgeries, stent insertions and hospitalizations for severe chest pain -- in those assigned to take Vytorin". All these events are not the same. People would care if this was an 1% reduction in death, heart attacks or strokes. But when these combined endpoints are used it is often just a decrease is "hospitalizations". If this decreased hospitalizations by 1 out of 100 over 6 years for chest pain that is much less important. Doc James (talk · contribs · email) 17:12, 18 November 2014 (UTC)

I'll just add my usual contribution: "There was this trial" is not biomedical information, and therefore MEDRS doesn't apply. You can support a claim that some people (in this case, some researchers) did something (in this case, ran a clinical trial) with exactly the same type of source that you would use to report a claim that some people (e.g., some politicians) did something (e.g., voted for some law). "We had a trial!" is not biomedical information. With luck, the trial will produce biomedical information, but the fact of the trial's existence is not biomedical information.

(If you all think it would help, I could add a table showing several examples of biomedical vs non-biomedical information to MEDRS.) WhatamIdoing (talk) 06:34, 19 November 2014 (UTC)

"All these events are not the same. People would care if this was an 1% reduction in death, heart attacks or strokes." That's exactly what it was Doc. The language of the press release states: "VYTORIN reduced the incidence of the composite endpoint of death due to all causes, major coronary events, and non-fatal stroke; this endpoint occurred in 38.7 percent of patients taking VYTORIN and 40.3 percent of patients taking simvastatin only (hazard ratio of 0.948, p=0.034)." Formerly 98 (talk) 19:22, 24 November 2014 (UTC)

PDQ, CKS and PubMed Health?

I have queries on three topics which I don't think are currently mentioned in the guideline:

PubMed Health

PubMed Health is an NLM search engine for evidence-based clinical information curated by Hildabast et al [26]. Its merits for identifying genuine systematic reviews/meta-analyses and sometimes providing links to DARE reliability appraisals have been discussed on Med project pages (eg here), but I can't see any mention yet in MEDRS itself. Worth citing under "Searching for Sources" and/or "Assess evidence quality"?

PDQ etc

PubMed Health also searches for evidence-based Clinical Guides, including the PDQ Cancer Information Summaries (in "Health Professional" and "Patient" versions) curated by the NCI. Worth a mention under "Other sources"?

CKS

Likewise, how about NICE's [.nice.org.uk/#?char=A clinical knowledge summaries]? [afaik, not searchable via PubMed Health]

109.157.83.50 (talk) 22:41, 23 November 2014 (UTC)

CKS is only available in the UK so IMO should not be used. Doc James (talk · contribs · email) 13:03, 25 November 2014 (UTC)

James, not sure what you mean by "only available[?] in the UK". Do you mean that it can't be accessed outside the UK? 31.49.13.194 (talk) 17:53, 25 November 2014 (UTC) (ex-109.157.83.50)

Yes, a message on the screen says "Unfortunately this site is only available from Great Britain. We apologise for any inconvenience." Yobol (talk) 18:15, 25 November 2014 (UTC)

Ah, thanks, that surprises me somewhat. A real shame, imo. 31.49.13.194 (talk) 18:59, 25 November 2014 (UTC)

• I've written to NICE about this (c/c to James). 31.49.13.194 (talk) 20:01, 25 November 2014 (UTC)

I am under the impression that the difficulty of accessing a source is not supposed to be a factor in deciding whether or not to accept it. WhatamIdoing (talk) 04:44, 26 November 2014 (UTC)

I have to agree with WhatamIdoing. If it can be considered a truly high quality source accessibility shouldn't stop it from being recommended or used. I think there are resources available for verification when needed. - - MrBill3 (talk) 08:45, 26 November 2014 (UTC)

Predatory journals

Many exist. Impact factors can actually be bought. I have no idea how many of these journals are indexed on MEDLINE. However, FYI, a paper called Get Me Off Your Fucking Mailing List, and consisting of nothing but 12 pages of that repeated phrase, along with some graphs of it and a process line, has been published by the distinguished-sounding International Journal of Advanced Computer Technology. And not as a joke. No human ever set eyes on it, and the e-journal took the money to do it, and did it. Wups. It's a bit like the 1996 Sokal Social Text Sokal affair, but this time highlighting the absence of ANY review, not just lack of competent review.

http://www.vox.com/2014/11/21/7259207/scientific-paper-scam

SBHarris 00:45, 27 November 2014 (UTC)

read the page until the end. See: "Correction: This article previously said the article was published by the journal. It was only accepted, because the author didn't want to pay $150."

It is still a good story, anyway.

Jeanloujustine (talk) 18:11, 29 November 2014 (UTC)

That's been a fun topic to watch as it's made the rounds on different sites. Predatory journals are indeed a problem, but can we really do anything aside from cautioning that they exist like we currently do on the page? Maybe if we have a reliable source saying the journal is predatory, lacks appropriate review, etc. we could call that journal article unreliable, but otherwise it's not really something we as Wikipedia editors can distinguish on our own. Kingofaces43 (talk) 02:24, 27 November 2014 (UTC)

It's worth checking Beall's List when a journal seems iffy. Alexbrn ^{talk|contribs|COI} 04:05, 27 November 2014 (UTC)

Shall we centralize discussions here: Talk:Predatory open access publishing? Fgnievinski (talk) 01:28, 29 November 2014 (UTC)

proposed change to lead

Beland offered 2 changes to the lead of MEDRS in this dif, namely:

• " To avoid people getting hurt, it is vital that the biomedical information in all types of articles be based on reliable, third-party, published secondary sources and accurately reflect current medical knowledge."
• and a bit later, added Primary sources should generally not be used for medical content. Many such sources represent unreliable information that has not been vetted by review studies, or preliminary information that will not bear out when tested in clinical trials."

I reverted. I think the 2nd change is mostly fine, but the first change about "to avoid people getting hurt" goes too far in a way that I struggle to define. I think where i am coming from, is a liability question. I know we have the general disclaimer but I don't think we want to acknowledge that people can get hurt if they read bad information (heck any information) in WP and act on it, and get hurt. Maybe I am wrong. I generally like clarity very much but this seems off somehow. Jytdog (talk) 19:50, 20 November 2014 (UTC)

But we *are* concerned about people getting hurt by wrong information; that's the whole motivation here. I don't think adding the third clause to what boils down to "don't rely on what I'm telling you to do; I know people do sometimes; but if what I say is wrong you might get hurt" increases liability over medical harm to the WMF or to individual editors, but if there's a legitimate worry that's a question for WMF lawyers. -- Beland (talk) 20:02, 20 November 2014 (UTC)

i hear you! interested to see what others will say. Thanks for weighing in directly! Jytdog (talk) 20:11, 20 November 2014 (UTC)

How about "to avoid people being misinformed"? Doc James (talk · contribs · email) 20:25, 20 November 2014 (UTC)

respectfully, Doc James that is a duck. How do you feel about the very clear statement? (which I agree is our concern!) Jytdog (talk) 20:29, 20 November 2014 (UTC)

I do not like it. Doc James (talk · contribs · email) 20:38, 20 November 2014 (UTC)

that is what my gut says too. am trying to articulate why... Jytdog (talk) 20:44, 20 November 2014 (UTC)

Jytdog I do not like it because it is sensational beyond the norm. It emphasizes that the pain of using poor medical information is "being hurt", when in all other communication channels the risks are typically downplayed. By being sensational here, the presumption is that for some reason additional warnings and scare tactics are needed on Wikipedia, presumably because Wikipedia is more dangerous than other options. The consensus in Wikipedia from the medical disclaimer discussions and the rest is that Wikipedia's health content does at this time not merit extraordinary and unusual warnings beyond what it already presents. Because of this past consensus, and because this is an extraordinary warning not used in comparable health informational channels, this kind of language is inappropriate. Blue Rasberry (talk) 15:46, 25 November 2014 (UTC)

wow Bluerasberry that is well thought out and very helpful! Thank you!! Jytdog (talk) 15:55, 25 November 2014 (UTC)

That's wrong, Bluerasberry. There is no consensus either way - either here or in the wider community - about the use of prominent disclaimers on our medical assertions. As there is no consensus, we default to the status quo. I think that if we were to run another RfC with a more general question ("Should we have a more prominent warning/disclaimer on our medical articles?" or similar) it would pass. In the last one there was a lot of concern that people may put more trust in our (particularly our more obscure) articles than they deserve. The difficulty was in deciding on a form of words. --Anthonyhcole (talk · contribs · email) 10:04, 30 November 2014 (UTC)

Anthonyhcole I cannot say what would happen with another RfC. There are multiple interpretations of the discussion in the last one. I saw what I expected to see in that discussion. Blue Rasberry (talk) 18:48, 30 November 2014 (UTC)

Many of "us" know where I stand on the issue (Wikipedia:WikiProject Medicine/RFC on medical disclaimer), and I don't think that changing this page is the way to effect the kind of change I advocate for ... so ... meh ... I'll skip this discussion. SandyGeorgia (Talk) 20:36, 20 November 2014 (UTC)

and for those who are not "us", i believe that Sandy's position is that we should have medical disclaimers on every health-related article. would be interesting to hear your reaction to the proposal though, outside of the way i framed it. do you like it? would it help people understand better why we are more strict about sourcing? Jytdog (talk) 20:40, 20 November 2014 (UTC)

There is so much more than avoiding "people getting hurt". I think it's covered on the RFC, and rehashing it will just hasten my (next) departure :) :) SandyGeorgia (Talk) 20:53, 20 November 2014 (UTC)

laughing. ok ok! we want to keep you as long as we can this time :) For those who want a refresher that RfC is here Jytdog (talk) 20:55, 20 November 2014 (UTC)

OK, if multiple people don't like the first part and can't explain why, I'll drop it. I'm on the verge of going around slapping tags on articles with inadequate citations for medical claims, so I'd rather spend my time and energy on that. Can I put the second part back? -- Beland (talk) 19:59, 21 November 2014 (UTC)

Beland I took a shot at adding the 2nd sentence back.... let's see if it sticks. Jytdog (talk) 20:30, 23 November 2014 (UTC)

Thanks! -- Beland (talk) 16:22, 24 November 2014 (UTC)

Source of the article : Wikipedia