Should we rethink our policy on story identification questions / identification requests?

Question

Almost a year ago, verbose asked How do we ensure that questions on this site attract scholars and enthusiasts? In my second answer to that question, I wrote that identification-request questions are a type of questions that dilute our site's focus.

Our current policy is to accept such questions; the highest-voted answer to the question What do we think about story identification? (currently 29 votes) also says,

You don't have to go out of your way to shut down ID questions as soon as they spring up, as long as they're rare and well-written.

We have learnt several things about this question type since that policy was defined in 2017:

1. This type of question is by no means rare: we currently have 603 such questions (and 27 that have been deleted, making identification-request the third-most-popular tag on the site.
2. People who post such questions often need some prodding and a link to the identification-request wiki before a question is edited into a shape that makes it answerable and acceptable.
3. It is a type of question that is often answered without reading the work being asked about; instead, good internet searching skills tend to be sufficient.
4. It is a type of question that typically benefits only the person who asked it, unlike, for example, questions asking about the history of literature, the historical background to a specific text and explanations of specific difficult passages.
5. In spite of the limited usefulness of these questions, they often get more votes than questions that actually teach us something about literature (see the types mentioned above).

Because of the last three issues, I doubt that this type of question makes a great impression on scholars and other serious readers of literature. For this reason, I suggest that we stop accepting this type of question. If we are serious about attracting scholars and other "students of literature", and the 10 upvotes on How do we ensure that questions on this site attract scholars and enthusiasts? suggest that we are, we should take seriously consider such a change in policy.

Answer 1

We shouldn't be pushing away one set of users in favor of another.

---

Do we need to be attracting scholars and enthusiasts? Yes, of course. Should we be pushing away amateurs and people who aren't "scholars and enthusiasts"? Absolutely not. Instead, we should be focusing on supporting both groups of people.

When I first joined the site, I wouldn't fall into that "scholars and enthusiasts" category; I was somebody who enjoyed reading, but didn't have any prior experience in analysis or any type of literature study. I became exposed to it through this site, participating as a reader.

Expanding on that, I think our userbase can be divided into four rough categories: Experts, enthusiasts, readers, and fans.

• Experts are those who are experienced in serious literature study, such as someone with a degree in Literary Studies or someone self-taught enough to qualify. They can provide answers to difficult questions, break down texts, shed light on history, etc.

• Enthusiasts are those who are interested in the field but aren't necessarily experts. They can ask questions about literary topics, provide some answers to literary questions, and are becoming more experienced with the field.

• Readers are people who aren't necessarily interested in analysis or theory or understanding every last metaphor; they're there because they enjoyed the story. Plot questions and story-IDs tend to be asked by this group.

• And fans are those who have a passionate interest in one or more works - like most of the contributors on Scifi.SE. These are the people who usually wind up asking authorial intent questions. There are very few of these at this point in the history of our site, but there were more when we first started.

I don't think any of these group of users are at odds with each other in using the site. Anybody should be able to ask a question and receive an answer; the "seriousness" of the subject matter shouldn't necessarily be a factor.

One of the goals of our site, in my mind, is to move people along from one group to the next. I arrived as a reader; I'd consider myself an enthusiast at this point (albeit focused more on the question side than the answer side). It's healthy for new users to arrive through one type of question, stick around, and find an interest for others. If we push away all our readers, then our pool of potential enthusiasts shrinks drastically - and then so does our pool of potential experts.

But even without that development of becoming more interested in "serious" literature study, we should not be prioritizing one set of users over another.

You make the assertion that story-ID questions drive away experts and enthusiasts. I would ask that you please provide some support for this claim; where have you seen that this is what's keeping experts away?
Counting deleted questions, story-IDs make up 630 of 6,187 questions. That's just over 10%, which, while not insignificant, means that 90% of our questions are not story-IDs.

On the other hand, we have experienced in the past that trying to prioritize "expert" questions drives other users away. I'm not interested in going through another period where half of our users feel like they're being told to leave by other users. That'll just negatively impact our site and lose a large percentage of our user base. A ghost town is not inviting.

---

Instead of trying to limit questions by readers, such as story-IDs, let's focus more on creating the content that will attract enthusiasts and experts. We can support all types of literature enthusiasts on the site, not just those interested in serious literary studies.

Answer 2

The question links the policy on story-identification questions to the attractiveness of the site for scholars and enthusiasts, without making an argument justifying this link. The title is phrased with an "if":

If we want to attract more scholars and enthusiasts, should we rethink our policy on story identification questions / identification requests?

The implication of the title is that rethinking the policy will attract more scholars and enthusiasts. But the body of the post does not justify or argue for the claim. What evidence do we have that the claim is true?

As I said in my answer to a previous question based on a similar assumption, it is risky to proceed on the basis of a claim like this, because the claim might not be true. Before deciding to close a whole category of questions in the hope of attracting new users to the site, it would be prudent to investigate whether this strategy has any prospect of success. Who are these "scholars and enthusiasts" who would like to contribute to the site, but are put off doing so because of the existence of story-identification questions? What evidence is there that these people exist?

Answer 3

My response to Mithical's answer is too long to fit into a few comments, so I'm taking the liberty to post it as an answer. There are several issues with their answer.

1. Its division of the userbase into four categories doesn't take into account how small the category of "experts" actually is; this group probably constitutes less than 1% of our userbase.

2. It misrepresents the suggestion about site scope as one about playing out one part of the userbase against another: "We shouldn't be pushing away one set of users in favor of another", "I don't think any of these group of users are at odds with each other in using the site", "we should not be prioritizing one set of users over another". Questions aren't users and users aren't questions. There is no such thing as a type of user who can only post identification requests. There have been users who have posted only identification requests, but let's not confuse accidental with inherent characteristics.

3. According to Mithical I claim "that story-ID questions drive away experts and enthusiasts". To drive away means "to force someone or something to leave". I'm not claiming that these questions drive away experts in this sense but that they prevent experts from taking the site serious. You can't drive away people who aren't here in the first place.

4. Mithical says identification requests represent "just over 10%, which, while not insignificant, means that 90% of our questions are not story-IDs". But take a look at the home page. When you load the homepage in a desktop browser, it displays 48 questions, which means that a new visitor would on average see four or five identification requests. (In fact, at the time of writing, it displays nine identification requests.) That's a lot of content that is helpful only to the users who posted those questions.

5. "where have you seen that this is what's keeping experts away?" There is no evidence for this. However, the almost complete absence of experts is visible both in answer quality and in voting behaviour: many answers that required careful analysis or a considerable amount of research get fewer upvotes than answers that were written up in a matter of minutes. That's not the sort of voting behaviour that would inspire confidence from experts who are considering whether to join the site or not. In addition, identification requests are not a type of questions I have ever seen in scholarly literature.

6. The link "trying to prioritize "expert" questions drives other users away" doesn't point to a scope discussion but to a post from October 2017 (more than four years ago!), where the OP said,

I can't honestly recall the last positive interaction I've had on the site. Nearly everything I've done here has been frustrating, anxiety-inducing, and unrewarding, especially recently.

Let's not draw false analogies between the current site scope question and ... a description that in my opinion no longer applies to our site.

7. Mithical says, "let's focus more on creating the content that will attract enthusiasts and experts". But this is the chicken-and-egg problem I'm trying to address here. Questions that would make the site more attractive to scholars are less well received than identification requests, even though the latter type of questions benefit only the question asker. In other words, those who "[create] the content that will attract enthusiasts and experts" are not rewarded to the same extent (or sometimes not at all) than those who post identification requests. What Mithical's answer comes down to, in my opinion, is maintaining the status quo, which implies keeping the site just as unattractive to scholars as it currently is.

8. Mithical claims that, "One of the goals of our site, in my mind, is to move people along from one group to the next." but provides only anecdotal evidence to support this claim. The theory that users who start out by posting identification requests later become regular contributors is not supported by any data. It is possible to verify the theory, though. Let's take a look at identification requests posted in 2020 and see how many users who posted such questions later became regular contributors. (Those who posted such questions last year may not have head sufficient time "to move (…) along from one group to the next".)

1. book about children who go to live in the woods by Joshua Barriger: user's only contribution; last seen early 2021.
2. Christmas/Saint Nicholas book by spikey_richie: user's only contribution; last seen late 2021.
3. Humorous novel about a man in a religious order who … by Sean Duggan: regular contributor who has posted 35 questions and 84 answers so far.
4. story "I did not sell my child" … by Shashank Sabniveesu: user's only contribution; last seen late 2020.
5. Book involving a child whose mother breaks into his piggy bank … by Sean Duggan: regular contributor, see above.
6. What is the children's book about a troll looking for a rope to hang his brother? by BirdNerd: user's only contribution; last seen late 2021.
7. What book involved forging one of the United States' founding documents? by AnonymousAsker: user's only contribution; last seen August 2020.
8. Book about a panda who ate liquorice and used a sudoku square to do magic by Helen (unregistered): user's only contribution; last seen August 2020.
9. Book about a mathematical British captain by Tygan: one of three questions contributed by this users, all of which were identification requests. (Their third and last question was deleted by the Community bot.)
10. A book by an American ex-spy … by Vladimir Markiev: the user's only question; they also contributed one answer by have not been seen since late 2020.
11. Novel about a Kazakh train station manager by gris_martin: user's only contribution; last seen September 2021.
12. Identify book about a guy in his late teens/early twenties who goes to live with a family … by guest123 (unregistered): user's only contribution; last seen October 2020.
13. Book about kids riding a flying snake in a magical realm by Tygan (see above).
14. a woman missing the train and living among strangers by Jim: user's only contribution; last seen October 2020.
15. Novel about a man who voluntarily has himself locked in another man's basement by Taylor Burke: user's only contribution; last seen November 2020.
16. Multipurpose furniture shapeshifts at inconvenient times, abandoned in attic by Santhosh J: user's only contribution; last seen December 2021.
17. Help identifying a 90s children's horror/adventure story featuring an abandoned house, … by Beanman: user's only contribution; last seen November 2020.
18. Children’s stories about motocross by cmbuckley: user's only contribution; last seen December 2021.
19. book where a boy starts volunteering at a vet's office after school by Chris Heilman: user's only contribution; last seen May 2020.
20. teenage girl finds out from her younger sister Martha that she was adopted by Magneto: one of two contributions by this user; both identification requests; last seen June 2021.
21. Poem criticising students' tendency to try to pry the one true meaning out of a poem by Tsundoku: question posted as implicit criticism of the tendency identified in the title, i.e. not as a genuine identification request.
22. Mid-1990s novel about young programmer who gets into trouble in Dallas by user10139 (unregistered): user's only contribution; last seen May 2020.
23. book where the main character is released from prison in order to be sent to conduct espionage in Italy by Scoots: user's only contribution; last seen August 2020.
24. Hardback book, read in the late 1980s, involving a wandering samurai investigating a cat demon … by Sean Duggan (regular contributor; see above).
25. novel about a woman who goes into deep introspection while stranded away from home by Carl Witthoft: user's only contribution; last seen May 2021.
26. Story about a beached fishing boat and the Chinese Cultural Revolution by Michael: one of two questions by the same user; the other one is about translation; last seen July 2021.
27. Horror story about boy being turned into a mannequin by Najonathan: user's only contribution; last seen December 2021.
28. Children's book about a stuffed dog and stuffed cat who eat each other … by 360Genius: user's only contribution; last seen April 2020.
29. Author of books about Mexican guru by Stephen Collicoat: user's only contribution; last seen April 2020.
30. story of a man who strapped a magnet to his forehead, has visions by Elliot Rocha: user's only contribution; last seen May 2020.
31. Horror book involving a book that when read the reader believes they are a character in that book by Daniel Burke: user's only contribution; last seen June 2020.
32. Popular informal physics book containing a quote from Lewis Carroll by Niklas Rosencrantz: user's only contribution; last seen July 2020.
33. Story About a Boy Lost in the Wilderness by Ron Maupin: the only question by a user who also posted one answer; last seen July 2021.
34. 1960s kids book with "invisible" dust … by Daniel: the only question by a user who also posted one answer; last seen November 2021.
35. Short children's book about a Mexican boy who likes to make clay horses by Sean Duggan: regular contributor (see above)
36. children's book about hobbit-like people by Katie (unregistered): user's only contribution; last seen November 2021.
37. Sentient manta rays living in Jupiter? by Alex Schultz: user's only contribution; last seen September 2021.
38. Story about a European captain marrying a tribal woman, … by Rakesh: user's only contribution; last seen February 2021.
39. Short story on British radio about fish gutters turned contract killers by Rand al'Thor: the site's top question asker (see above).
40. Identify an Archaeology Romantic Suspense or Mystery Novel - with nymphomaniacs! by HRingler (unregistered): user's only contribution; last seen September 2020.
41. Book about people cut off from rest of the world that don't kill animals by Caleb Mauer: user's only contribution; last seen August 2021.
42. Is there a version of Beauty and the Beast where the bad sisters get punished? by JJJohn: user's only contribution; last seen August 2020.
43. A feather chokes an opera singer to death on stage by Kevin Ryan: the only question by a user who also posted 7 answers (included one that was deleted), all in 2020; last seen September 2020.
44. fiction novel about Atlantis and digital or analog "discs" by Jason Weinzierl: user's only contribution; last seen April 2021.
45. book involving a boy and carnival by The Social Hermit: user's only contribution; last seen October 2021.
46. Girl in a marshy village setting whose friend is small with big feet by Popplio Lover: user's only contribution; last seen May 2021.
47. Novel with a spy/assassin who helps the grand-daughter of a Nazi hunter, … by Sean Duggan: regular contributor (see above).
48. Book where all the characters are named after chess pieces by sam_smith: one of four questions, all of which were identification requests (including one that was deleted); the user also posted a deleted answer; last seen December 2021.
49. Book with stories continued on later pages with at least one misleading illustration … by Sean Duggan: regular contributor (see above).
50. Young adult fantasy book about a girl with gray eyes and powers related to metal by soup bar (unregistered): user's only contribution; last seen July 2020.
51. Book set during the Black Death: nine travellers each with a secret by Rand al'Thor: the site's top question asker.
52. Story in which a boy's mother dies, he forgets about it … by joss5673: user's only contribution; last seen September 2020.
53. Looking for a book whose title roughly translates to Way to Oregon by TulakHord: user's only contribution; last seen Augst 2021.
54. Character doesn't understand "question mark" dance, … by Silver: user's only contribution; last seen October 2021.
55. Young-readers book of short-stories from 1960s or 70s - "Teenaged Millionaire"? by Weaky Reader: user's only contribution; last seen September 2021.
56. Crime novel with gay policeman as main character by [account deleted in 2020]: user's only contribution.
57. author of a story about a tourist who got a tattoo from a renowned artist by kosmos: the user's only question, which they also answered; no other contributions; user last seen in September 2021.
58. Short Story with "All Odd Numbers Are Prime" Joke by Mike Stone: one of three questions, two of which are identification requests; the user also contributed one answer; last seen September 2021.
59. Crime drama/mystery where a woman suffers a car crash, and searches for a child who doesn't exist by Sean Duggan: regular contributor (see above).
60. Children's book with a lost treasure in a hidden valley only accessible by swimming underwater by Craig (unregistered): user's only contribution; last seen January 2020.
61. American comics strip about mathematically shaped pancakes by WoJ: one of three questions by the same user (most recent one in January 2021); last seen October 2021.
62. Roman short story about a senator and a slave by 4gus71n: user's only contribution; last seen August 2020.
63. SF short story about a hotel frozen in time by juan s (unregistered): user's only contribution; last seen August 2020.
64. Can you identify this poem about axolotl, fish, salamander by Tilly: the only question by a user who also posted a now-deleted answer; last seen October 2020.
65. Story about two men walking through the forest by sam_smith: see above.
66. sci-fi short story: three robots sent as ambassadors to prevent an interplanetary war by Lindsey (unregistered): the only question by a user who also posted a now-deleted answer; last seen October 2020.
67. Gore Vidal’s book about the Mexican-American war: was anything published? by tale852150: one of two questions posted by this user; last seen September 2021.
68. poem about life as a test by Keshav Srinivasan: one of three questions posted by this user; most recent one in November 2020; no answers contributed; last seen January 2021.
69. Story/article about the 1969 moon landing's meaning to a black man by Giorgio Lauri (disabled/inactive account, so presumably user's only contribution).
70. Short story about a bullied boy winning a rock throwing fight … by Newbie12345: user's only contribution; last seen February 2020.
71. Young Horror Anthology - Short story with devices passed into a hole in a wall, returned powered by worms...? by p.q: user's only contribution; last seen March 2020.
72. Possible Jerome Salinger short book about hockey player and his girlfriend by imlookingforabook (unregistered): user's only contribution; last seen March 2020.
73. 19th-century French "flash fiction" writer/journalist by Denkof Zwemmen: the only question by a user who also posted 17 answers. The question was their first contribution.
74. Short story about a drug addict in a prison in Mexico by zovits: user's only contribution; last seen December 2021.
75. Comic about Mayas and/or Aztecs invading Europe … by Tsundoku: regular contributor.
76. story about Japanese storyteller's apprentice who is mistaken for a master by Simon K: one of two questions by the same user; no other contributions; last seen March 2020.
77. What novel is Yehsao Paoyen? by Tsundoku: regular contributor.
78. children's short story about boy flying plane in storm by Curious One: user's only contribution; last seen March 2020.
79. What story is the idea that it's better to pick a small box rather than a large one from? by Angelos: user's only contribution; last seen December 2021.
80. Author and title of poem: "Follow, follow me into the South, ..." by Scott: one of two questions by the same user, both of which are identification requests; last seen April 2020.
81. Trying to remember a poem that I only have a snippet of by Scott: see above (previous question).
82. Short fiction in which the suits of playing cards (…) are used in place of some words by J Sheep: user's only contribution; last seen December 2020.
83. R. K. Narayan's maiden speech to the Indian parliament? by Tsundoku: regular contributor.
84. Comic Book about Jupiter and time travel by Bruno Vilela: the only question by the user, who self-answered it; last seen July 2020.
85. Does anyone recognise this poem - from my mum (90)? by Wilbur: user's only contribution; last seen March 2020.
86. mystery novel series whose protagonists are two old retired women by clef: the only question by the user, who self-answered it; last seen January 2021.
87. What short story has a soldier deliver a secret message like “shoot the messenger”? by Paul Chernoch: user's only contribution; last seen December 2021.
88. U.S. soldier mistaken for football star in Japan by nebogipfel: user's only contribution; last seen July 2021.
89. US Nebraska middle school literature texts from the 70's by B. Goddard: user's only contribution; last seen December 2021.
90. Short (possibly children's) story set in Norfolk glorifying the English Countryside by Lt. Commander. Data: user's only contribution; last seen September 2021.
91. Short Story About a Mystical Bog and Illicit Affair Involving a Professor by CouchFort: user's only contribution; last seen August 2020.
92. Short story in which notorious safe-cracker retires but … by ZachTalonGamesYT: user's only contribution; last seen September 2021.
93. develop old film, solve an old mystery by Pete: one of three questions by this user, who also posted 15 answers.
94. Short story: an executioner, uncaring to convicts, discovers a watch that he gifted to his son by abbe: user's only contribution; last seen May 2021.
95. Short story - man obsessed with news, then lives solitary life, … by Slow'R Reader: user's only contribution; last seen December 2020.
96. Info about "Wisdom is One" by Benjamin Williams Huntsman? by Quisizyx: user's only contribution; last seen October 2020.
97. Short story about humans that portrays them as savages by AHolasek: user's only contribution; last seen June 2020.
98. Man Slowly Dying From Internal Injury by jwBurnside: user's only contribution; last seen June 2021.
99. kid who waves at the train by Mat J: user's only contribution; last seen November 2021.
100. play 5-10 years ago where all the dialogue consisted of generic words describing what might be said by scottef: user's only contribution; last seen May 2021.
101. Title and Poet of Post WWI poem closing with … by Jim: user's only contribution; last seen March 2021.
102. Poem about a joker with disorganized attire by ShivCK: user's only contribution; last seen October 2021.
103. What song is mai chhoro brajaki basari? by Tsundoku: regular contributor.
104. English title of Tolstoy's memoirs "Without love, It's easier to live"? by Dmitrii Pisarenko: user's only contribution; last seen July 2021.
105. Did Julian Huxley write a poem about an axolotl …? by Phil van Kleur: user's only contribution; last seen October 2020.
106. poem by Wisława Szymborska by Deborah W. (unregistered): user's only contribution; last seen September 2020.
107. Japanese story Joseph Campbell tells of a samurai spat upon in The Power of Myth by Vincent Noto: user's only contribution; last seen December 2021.
108. WWII (?) comic about a spy doing propaganda for the bad guys by Fizk (disabled/inactive account, so presumably user's only contribution).
109. Mystery short story where the owner of a dog is accidentally murdered via strychnine by Sean Duggan: regular contributor (see above).
110. Tale of a Dog Chasing the Sun/Trying to Find Its Owner? by North Læraðr: one of 17 questions by this user, who also posted 15 answers; last seen November 2021 (last contribution in February 2021).
111. Charles Bukowski's poem about waking up every morning and trying to be a poet/writer by hod: user's only contribution; last seen December 2020.
112. What was the case where Sherlock answers Watson's private thoughts? by ArcaneLight: user's only contribution; last seen September 2021.
113. In what story of Régnier does a man commit suicide without understanding the reason? by MaudPieTheRocktorate: user's only contribution; last seen November 2021.
114. short story about a short irrefutable self-evident proof that God exists that is kept secret by 今夜九: one of three questions contributed by this users, who also posted one now-deleted answer; last seen October 2021.
115. detailed-prose book of Aesop fables by hattie (unregistered): user's only contribution; last seen July 2020.
116. Roald Dahl short story - wife kills drunkard husband by spiking his drink by Kirsty: user's only contribution; last seen January 2021.
117. Is this a real Indian fairy tale about a monkey who changes colour and marries a princess? by Rand al'Thor: the site's top question asker.
118. original Persian version of these Rumi poems from "The Essential Rumi"? by Inaam Nadeem: one of two questions contributed by this user, both identification requests; last seen January 2020.
119. Horatio Hornblower book as part of anthology? by user11674 (unregistered): user's only contribution; last seen December 2020.
120. What book of Eugène Sue's 'The Mysteries of the People' … by Sven3B: one of two books contributed by this users; last seen September 2021.

In 76 cases, the identification request was that user's only contribution. This includes 13 questions from users whose accounts are marked as "unregistered". There are a number of questions by regular contributors, but these are not users who moved from identification requests to other types of contributions (Sean Duggan, Rand al'Thor, North Læraðr and myself). Besides these users, there are a few who have posted a few other questions or answers or both but these contributions were almost always limited to 2020 with no continued participation in 2021. It is quite clear that most identification requests are just one-off "drive-by" questions by people who are not interested in making other contributions. In conclusion, the data from 2020 do not support the claim that users who start out with identification requests later "move (…) along from one group to the next"*. Changing the site scope to exclude identification requests would reduce the number of one-off "drive-by questions" but not have a significant negative impact on the base of regular users.