Quality of Information: For PinkFreud Best answer on the web
help me out with my question about software for pricing inventory
(#182608) and saw that no one has (why not? I wonder--I thought that
would be an easy one, straight facts)--and that is when I saw that you
had answered one of Granny’s marvelous survey questions by listing me
among your favorite customers. Wowee! What a thrill! I am still
grinning. To be on anybody’s list would be a real honor and a
tremendous compliment. To be on yours, dear Pink, with your
superlative standards of research and delivery, your remarkable
presence, and your polished-to-gleaming style, is like getting the
prettiest valentine in the class--and a dozen roses besides.
To say thank you, I’d like to offer this question to you.
Please be patient with me, now. This is going to take a minute.
When I first got the hang of web-surfing, just a little over two years
ago, and tried out various search engines for the first time, I was
awed, as youngsters can never be who take it for granted, at the
depth, breadth, and sheer mass of information and opinion and creative
work that can be found on the Internet. I still have a deep love of
books and libraries and traditional scholarship (and wooden
card-catalog drawers you can browse in), but the magical allure of the
web is almost irresistible. I spent one whole Sunday afternoon just
reading the things that came up when I typed “BP” into Google and
clicked “search.”
Hard on the heels of that initial astonished excitement, however, came
a disturbing question: how does one evaluate the quality of
information found by this means?
To be sure, one might (and should) ask the same question of what one
finds in print. Plenty of worthless junk gets into books, magazines,
and newspapers and always has. Nonetheless, there is a process that
acts as a filter of a sort, and there are some barriers to wholesale
delivery of written material to the public that require effort to
overcome, such as the mechanics of production, the hurdles of
distribution, and the challenge of getting a prospective audience to
pay for your product. In the year or more that it takes a scholarly
book to go from an author’s hands through the acquisitions process and
the steps of developmental editing, author’s revision, manuscript
editing, reviewing, copyediting, production, proofreading, and
indexing, the content is subject to many rigorous tests. The
financial investment that the process requires and the dependence of
its success on acceptance by a community of prospective critics say
something about the level of confidence one might place in its merit.
By contrast, anyone can put anything on the Internet, at very little
expense and with no significant resources required beyond a personal
commitment of time and energy. There is no vetting of any sort, no
necessity of rigor or discipline to the content, no measure of quality
applied. What it takes to put content onto the Internet is simply
nothing like what it takes to bring a book or magazine to publication.
And the quality of much of what we can find on the Internet is
conspicuously wanting. It may not be too difficult to pick out the
defrauders, deceivers, and dissemblers who write pseudo-facts in
service of some ideology or cause; some quality of monomania, some
edge of hysteria or shrillness may give them away. But even if we put
these aside, along with the clever hoaxes that have no true harmful
intent but still may grossly mislead, we see the work of the unwitting
who perpetrate error out of sheer arrogance: the transcribers of Poe,
for instance, who render the line “as of someone gently rapping” as
“as if . . . ,” perhaps thinking to correct Poe’s impeccable grammar
with something that sounds more customary to their own ignorant ear,
or quoters of the allegedly Hippocratic “primum non nocere” who insert
a comma into the expression as if they were punctuating an English
sentence. We find the admiring students’ flawed renditions of a
teacher’s lessons imperfectly understood, or the postings of the
careless or foolish who simply cannot differentiate their own
perceptions from anything objectively verifiable and write reckless
nonsense masquerading as reporting. Until our educators have put
aside self-esteem as the paramount value in education and committed
themselves to rescuing discrimination from the bad rap it got in the
sixties, when it became equated with blind, unreasoning prejudice
although it actually means the opposite--a necessity not only for clear
rational thought but indeed for any sentient creature’s survival—-who
will show us how to tell good information from bad, to differentiate
between fact and opinion, to recognize propaganda, to question the
authenticity of data, and to reject specious logic? How can our young
people, in particular, mine the Internet’s amazing treasury of
knowledge and pick out the gold from all the dross? These are
questions that trouble me.
And so my question to you, PinkFreud, is this: in your online
research, what discriminators do you use to recognize sound
information, to test what is offered as fact, to weed out fringe
sources and dubious data and special-interest interpretations, to
validate the quality of the references you quote and cite? What
questions do you ask, or what cues do you look for, or what signs of
legitimacy, or what reliable authorities? How, in sum, do you judge
what is good information?
I'd tip you with Au if I could, but you'll have to settle for Ag. Find me in MN and I'll buy you lunch.
http://www.philosophy.unimelb.edu.au/reason/critical
Right after I finished posting my answer, I went to the forum where Researchers relax and share useful stuff with our Google Answers Researcher compatriots (we call ourselves GARs, which sounds kinda pirate-y. I must post my GAR pirate poem for you someday.)
When I arrived at the forum, two private messages awaited me. Both offered useful info related to your question.
My colleague luciaphile has shared with me this collection of links:
http://www2.vuw.ac.nz/staff/alastair_smith/evaln/evaln.htm
And my colleague tehuti offers this:
http://sosig.ac.uk/desire/internet-detective.html
Thanx, and a tip of the pink hat, to luciaphile and tehuti. One of the things that makes Google Answers really special is the team spirit among Researchers. Yay team! :-D
~Pink
Completely agreed with Pink (Factor #9 included).
I'd like to add that certain content is almost always questionable, even if otherwise I'd trust the source. Such as:
1. Quotes and their origin.
2. Something that sounds really unbelievable.
3. A "fact" that could hurt a company's public image, or could hurt an individual. 4. Someone wants to sell something.
5. A person or organization needs help of some sort.
6. A warning is issued, something that would somehow affect my health or PC. 7. Something is true but cannot be explained by science.
8. Conspiracies preventing this "fact" from being published.
9. A "fact" that makes you feel really good, or really bad.
10. It's a statistic.
1. Quotes and their origin.
There are so many misquotes running around online, you really have to validate the source, and find out to who many other people a buzz-word, phrase or quip has been attributed to. Think of the infamous "I created the Information Highway" (paraphrased) which is supposedly coined by Al Gore, and makes the 'Net-savvy crowd snicker -- however, he never coined it, nor did he _say_ he coined it. I won't even try to find any information related to it because, yes, it _might_ just be true he said it, or he was just quoted out of context, who knows! But if someone asked me for the fact only, when it comes to quotes, I'd make sure I find reliable sources.
2. Something that sounds really unbelievable.
If something is really, highly shocking, funny, but makes sense in a strange sense of "I knew it!" or "I was always suspecting that!", then probably it's a lie. The more a "fact" awes you the more likely you will tell it on, and when you spread a lie it becomes a fact to many people pretty soon, simply because they hear it over and over. Urban legends are part of this. If your gut-reaction is "I don't believe it", well, then don't! Better to try to track down the reality behind the myth.
3. A "fact" that could hurt a company's public image.
For instance at this moment there's an image making its circles through the email world. It's a sexist depiction of a couple wearing Puma shoes, to spare you the details (and the link to the image, as it's sort of adult-content). As soon as I got the email -- in the style of "Check out this new Puma advertisement!" -- I was not taking for granted that indeed the originator of this campaign is Puma (and indeed, they now officially distance themselves from the pictures). There's a certain mentality of paranoid, Orwellian "them-against-us" in popular culture, which helps spread bad news about the big ones in business.
Another variant of this is negative information about individuals. Someone might be interested to simply ruin this person's credibility in an act of revenge (especially after being "let off" from a job). And bad-mouthing is so much more effective online, since it's easy to publish information anonymously.
4. Someone wants to sell something.
Well, no need to elaborate on this one really. If someone has a strong interest that you believe her, there might be more behind her "facts" than meet the eye. I guess everybody knows that ads lie. Matter of fact we take it for granted, right? So if someone tells you his new soft drink will make you run faster than before, we simply reject this "fact".
5. A person or organization needs help of some sort.
These hoaxes are often send out via email by the innocent, but dangerously naive, people. The same crowd that double-clicks on links in their browser. Now when they find out that little Billy located somewhere in a romantic French village needs money (or loads of post-cards) to cure his sickness, of course they want to help! They do so by sending this message out to hundreds of others, and there's a good chance some of them (you guessed it, the link-double-clickers) will pass it on. Especially out of the instinct of saving money; "Well I can't help directly but I do my share by asking others to do so".
6. A warning is issued, something that would somehow affect my health or PC.
There's some funny emails that tell you to delete this or that file from your computer hard-disk immediately, or else bad things could happen. People who are naturally suspecting a virus everywhere will be so relieved they have been informed in time, they rush to open their Windows Explorer, and delete the file in question. Which might either be completely harmless, or of crucial importance to the Operating System -- and _not_ a virus at all.
So, if someone would ask me to research wether this or that file is harmful or not, I would check only with authoritative sources, and double-check their information.
7. Something is true but cannot be explained by science.
Really, I appreciate people who are into new age, healing crystals, and all that. They probably live a life much happier than the cynical ones. But if you don't even attempt to prove it, or only with methods that in themselves are not proven, it's plain dubious to me. That doesn't mean I cannot appreciate what I cannot understand or reason about (like art), it just means I don't count those things as belonging to the world of hard facts.
8. Conspiracies preventing this "fact" from being published.
Many hype their content by boasting about how this information has been locked up so many years by the government. In general, this would ring my alarm bell. OK, it might just be true, and I would certainly look into it if I have the time, but mostly conspiracy theories are just lies (entertaining ones, nevertheless).
9. A "fact" that makes you feel really good, or really bad.
People tend to believe what they _want_ to believe. Things that make them feel good. So much spam starts with "You've won the lottery".
And then we have those people who believe everything you tell them if it's bad news. Because they're born pessimistics, and always curious as to why they have been spared from bad fate so far. They will almost sigh with relief when they find out that sickness finally hit them ().
So, if a doctor tells me I'm dying tomorrow, _or_ that I don't have to worry at all, I'm highly suspicious. (And I never win in lottery, either.)
10. It's a statistic.
I don't really need to explain this one. There's a saying that statistics are like bikinis. ("What they reveal is suggestive, but what they conceal is vital.") I don't say statistics are always questionable in content, but yes, that one _has_ to question them to use them in research. Here especially public opinion votes, or product comparison tests. It's always important who sponsored the poll, who's publishing the comparison, and what other statistics are available on the subject. Some people would believe just about anything as long as it comes with some nice graphs and numbers.
And, since I already got carried away, I would like to quote from this Monty Python sketch (which also includes, yes, a statistic):
"Presenter: Good evening. Tonight 'Spectrum' looks at one of the major problems in the world today - that old vexed question of what is going on. Is there still time to confront it, let alone solve it, or is it too late? What are the figures, what are the facts, what do people mean when they talk about things? Alexander Hardacre of the Economic Affairs Bureau.
(Cut to equally intense pundit in front of a graph with three different coloured columns with percentages at the top. He talks with great authority)
Hardacre: In this graph, this column represents 23% of the population. This column represents 28% of the population, and this column represents 43% of the population.
(Cut back to presenter.)
Presenter: Telling figures indeed, but what do they mean to you, what do they mean to me, what do they mean to the average man in the street? With me now is Professor Tiddles of Leeds University...
(Pull out to reveal bearded professor sitting next to presenter.)
Presenter: ... Professor, you've spent many years researching into things, what do you think?
Professor: I think it's too early to tell."
I hope this helps.
~Pink
This is an interesting topic to me. Determining the quality of information is one of those things that I tend to do very quickly and almost without thinking about it.
Most libraries have prepared worksheets for their patrons on some basic things to look for in an internet based source:
1. Authority:
Who's presenting this information? What are their credentials? Do they have a bias? Do they seem reliable?
2. Affiliation:
Look at the domain name (.edu; .com; .gov, etc.). If it comes from an university, is it still a personal site? Are they selling something? Is there information about the group or person that created the page? Does the group or person have a bias?
3. Accuracy:
How are they backing up their information? If it's scholarly, is there a bibliography? Spelling and correct grammar are things I like to see, but there are instances (topics not well represented on the WWW) where I'll take what I can get--as long as I can supplement the information I find with something more substantive.
4. Currency:
How old is this information? Is there a date saying the site's been updated? Are there a lot of dead links? When was the page created?
5. Purpose:
What was the aim in creating the web page? Did that effect the content? I'm all for the objective and the unbiased, but there are times when motivations will be significant (e.g. an organization of sufferers of a particular illness or disease have a very personal motive driving them to get more information than say, some student who's put up a casual web page on the same topic for a class project).
6. Audience:
Who is this directed to? Sometimes it's useful to purposely look at sites designed for children--when I was learning HTML, for example, I found that those pages tended to help me a lot more than the ones meant for adult.
7. Why:
Something I notice a lot is that people often try and approach each and every search in the same fashion. That can be just as much a hindrance as a help. Each search you do (and this goes for print material as well) is different. It's good to think about why you're looking for information in the first place, because your criteria for inclusion will vary greatly depending upon your motivations.
Regards,
luciaphile-ga
It's nice to be noticed. :-)
In approaching the matter of the reliability of Internet materials, I'll give a few examples of online information that I found less than credible, and the reasons behind my judgments of these sites.
Sometimes information seems suspect because the authority of the originating source is doubtful, in the context of the Web site where the information is found. For instance, I would be a bit leery of quoting (as a sole authority) any site that uses a plenitude of animated graphics as decoration. Likewise, there's nothing quite like a blast of uninvited midi music to turn me off and make me want to dump a site from consideration. Pop-up advertisements are another warning sign that this is not the sort of site I want to inflict upon my Google Answers customers. While I have occasionally used source material from pages which committed the abovementioned abominations, I generally steer clear of sites that are filled with hyperkinetic dancing smiley-faces, midi songs that would wake the dead (or at least make them roll in their graves, if they were musicians,) and ads that tell me I have won something, or should punch the monkey, or have an unprecedented opportunity to spy on everybody in the universe with a handy-dandy webcam.
Here's a fascinating question that I tried to tackle last year; I ended up commenting without answering:
http://answers.google.com/answers/main?cmd=threadview&id=77755
I would have loved to have answered the question by citing an authoritative source that said the stringy things on bananas are called "phloem bundles." In point of fact, I think this is a plausible answer. However, I could not find this exact phrase elsewhere, and I hesitated to cite as an authority a person who identifies herself only as "armadillogirl" on her Web site.
Of course, much the same argument could be made against taking the word of someone who calls herself "pinkfreud." ;-)
Earlier today, I tackled a question regarding the reasons for my having chosen Leonardo da Vinci as the world's greatest genius ever:
http://answers.google.com/answers/main?cmd=threadview&id=191712
Since I didn't want to pull the entire answer out of my posterior, I looked for some interesting links with which to dress up my verbiage. Here's one site that I rejected:
Mind Publications
http://www.mindpub.com/art204.htm
I knew I was in trouble here when I came to this sentence: "He was a world class anatomist, acrhitect, artist, botanist, city planner, costume and stage designer, cher, humorist, emgomeer, hunoirst, engineer, equestrian, philosopher, and phuysicist." Sheesh. Anyone who doesn't even take the time to run a spell-check on his prose isn't a very impressive authority on "geniouses" (his spelling.) And what does Cher have to do with it? I bet she can't spell either.
In addition to decorative trappings and poorly-constructed content, another warning sign that a source of information may be less than reliable is that the site has an identifiable agenda, or takes a well-defined stance of some kind on a controversial issue, WHETHER OR NOT THE ISSUE IS RELATED TO THE INFORMATION I WANT TO QUOTE. (Please excuse my shouting in the last sentence, but it's difficult to place emphasis within the narrow constraints of GA's answer box, which does not permit HTML conveniences such as italics, bolding, or underlining.) If I perceive a Web site as being an issue-driven mouthpiece for propaganda, I am going to be skeptical of accepting the accuracy of *any* material I find on such a site, even if the material is not part of the hot-button issue that the site is promoting (or discouraging.) There have been several instances in the past when I was seeking information which led me to Web sites related to religious matters. Much to my alarm, I found that it is not uncommon for both pro-religious and anti-religious sites to misquote scripture from the world's great holy books. In addition, I've found numerous re-worded poems and much misattributed prose on "agenda" sites. Even if I happen to share the belief system of the site in question, my BS-detecting radar goes into high gear when I visit such a site; I would hesitate to use this kind of site as a sole authority on anything.
Humans have a tendency to accept information that is found in more than one source. Ever since I was a rather cynical child who learned that advertising was full of weasel-words, I've been cautious about assuming that seeing something in print a number of times means it is likely to be true. The Internet contains many statements of factual error, some of which are reduplicated an astounding number of times. One of my favorites: "A duck's quack does not echo."
Google Web Search: "a duck's quack does not echo"
://www.google.com/search?hl=en&ie=UTF-8&oe=UTF-8&q=%22a+duck%27s+quack+does+not+echo
Although you'll get 153 hits on this phrase, all the repetition proves is that quackery of another kind echoes quite a bit. Anyone who has ever lived near a pond where ducks tend to flock can tell you that the echoes darn near drive you crazy. I don't know who started this trivio-erratum, but it's pure malarkey, and you can take that to AFLAC.
My colleage larre-ga, one of Google Answers' very finest Researchers, took on the quack-echo question once, and I suspect that the customer's poor rating of the answer had to do with the infernal frequency of the misinformation that infests the Internet:
http://answers.google.com/answers/main?cmd=threadview&id=55003
Off the top of my head (which is where I keep such things,) here's a list of factors that can influence my judgment regarding the trustworthiness of Internet info:
1. Overall appearance of the site
2. Credentials (if any) of the site's author or Webmaster
3. Types of advertising that appear on the site (pop-ups are the
spawn of Satan)
4. Hosting of the site (generally, AOL = bad, .edu = good)
5. Proper use of English (with leniency for a site run by a
non-native user of English)
6. Apparent objectivity and lack of axe-grinding or propagandizing
7. Adequate citation of sources (citing only the author's name
won't cut it)
8. Links to other sites that are known to be reliable
9. There is no factor #9
10. Brownie points for good humor and wit (genuinely funny people are, in my experience, more careful with facts and tend to be more
trustworthy than are humorless wretches.)
I feel rather uneasy about preparing an answer that is mostly just me jabbering, and doesn't have any non-Google links in it, so here are a few good pages on the subject of evaluating the reliability of information:
Washington State University: Savvy tips for evaluating websites
http://wsutoday.wsu.edu/completestory.asp?StoryID=527
Forbes: For 53% Reliable Information, Click Here
http://www.forbes.com/technology/ebusiness/2003/01/31/cx_da_0131topnews.html
Johns Hopkins University: Evaluating Information Found on the Internet http://www.library.jhu.edu/elp/useit/evaluate/
Wellness Junction: Tips for Finding Reliable Information on the Internet http://www.wellnessjunction.com/athome/selfcare/trust.htm
Humboldt State University Library: Using Internet Sources
http://library.humboldt.edu/infoservices/literacy/module5/mod5inet.htm
This is the seach string that brought me the best results:
Google Web Search: "reliable information" + "internet"
://www.google.com/search?hl=en&ie=UTF-8&oe=UTF-8&q=%22reliable+information%22+internet
Thank you so much for asking my opinion on this most interesting subject. It's always a pleasure to be able to offer assistance to a customer who has a track record of being gracious, intelligent, and insightful (and whose name makes a super-terrific "Scrabble" word.)
* * * * * * *
My best to you,
Pink
I'm tickled "pink" to be asked for my view on an interesting subject. I am going to take my time preparing an answer for you. Gathering my thoughts sometimes is a process somewhat like herding cats, so I'll be working on this for a few more hours. Just wanted you to know that I'm not ignoring your question. ;-)
~Pink
There are so many misquotes running around online, you really have to validate the source, and find out to who many other people a buzz-word, phrase or quip has been attributed to. Think of the infamous "I created the Information Highway" (paraphrased) which is supposedly coined by Al Gore, and makes the 'Net-savvy crowd snicker -- however, he never coined it, nor did he _say_ he coined it. I won't even try to find any information related to it because, yes, it _might_ just be true he said it, or he was just quoted out of context, who knows! But if someone asked me for the fact only, when it comes to quotes, I'd make sure I find reliable sources. >>
This "quote" is indeed an urban myth; Phil Agre debunks it here:
http://commons.somewhere.com/rre/2000/RRE.Al.Gore.and.the.Inte1.html
Herding cats, of course, is easier if you know what they like to eat. The right treats are like a beacon to them.
Gee, is my face pink. ;-)
"...there's nothing quite like a blast of uninvited midi music to turn me off and make me want to dump a site from consideration. Pop-up advertisements are another warning sign that this is not the sort of site I want to inflict upon my Google Answers customers. While I have occasionally used source material from pages which committed the abovementioned abominations, I generally steer clear of sites that are filled with hyperkinetic dancing smiley-faces, midi songs that would wake the dead (or at least make them roll in their graves, if they were musicians,) and ads that tell me I have won something, or should punch the monkey, or have an unprecedented opportunity to spy on everybody in the universe with a handy-dandy webcam."
I think the midi files annoy me more than the pop-ups. The few times I've cited one of them as a source, I've warned the customer about the music. I am at a loss to understand why the designers of those sites fail to include a "stop" button for the music - they are sooooo simple to add.
#If you have any other info about this subject , Please add it free.# |