Datasets

The AdversarialQA Round 1 data consists of three RC datasets constructed using different models in the annotation loop; D(BiDAF), D(BERT), and D(RoBERTa). Each dataset consists of 10,000 training, 1,000 validation, and 1,000 test examples. The data is provided in the same format as SQuAD 1.1 for convenient processing and evaluation.

In order to understand the data better, we provide some examples from the validation sets of each of these datasets.

Examples from the Datasets


D(BiDAF)

In 2014, economists with the Standard & Poor's rating agency concluded that the widening disparity between the U.S.'s wealthiest citizens and the rest of the nation had slowed its recovery from the 2008-2009 recession and made it more prone to boom-and-bust cycles. To partially remedy the wealth gap and the resulting slow growth, S&P recommended increasing access to education. It estimated that if the average United States worker had completed just one more year of school, it would add an additional $105 billion in growth to the country's economy over five years.

How much more education would workers need to add billions to growth?

What was S&P's idea on how to deal with disparity?

What does S&P stand for?

Martin Luther married Katharina von Bora, one of 12 nuns he had helped escape from the Nimbschen Cistercian convent in April 1523, when he arranged for them to be smuggled out in herring barrels. "Suddenly, and while I was occupied with far different thoughts," he wrote to Wenceslaus Link, "the Lord has plunged me into marriage." At the time of their marriage, Katharina was 26 years old and Luther was 41 years old.

What was Katharina's last name?

In a letter who did Luther credit for his union with Katharina?

What did Luther say he was engrossed in?

In economics, notable Nobel Memorial Prize in Economic Sciences winners Milton Friedman, a major advisor to Republican U.S. President Ronald Reagan and Conservative British Prime Minister Margaret Thatcher, George Stigler, Nobel laureate and proponent of regulatory capture theory, Gary Becker, an important contributor to the family economics branch of economics, Herbert A. Simon, responsible for the modern interpretation of the concept of organizational decision-making, Paul Samuelson, the first American to win the Nobel Memorial Prize in Economic Sciences, and Eugene Fama, known for his work on portfolio theory, asset pricing and stock market behaviour, are all graduates. American economist, social theorist, political philosopher, and author Thomas Sowell is also an alumnus.

Paul Samuelson was the first of these?

Milton Friedman was a major advisor to what Prime Minister?

What alumni was a proponent of regulatory capture theory?

The concept of organization decision-making came from whom?


D(BERT)

BSkyB's digital service was officially launched on 1 October 1998 under the name Sky Digital, although small-scale tests were carried out before then. At this time the use of the Sky Digital brand made an important distinction between the new service and Sky's analogue services. Key selling points were the improvement in picture and sound quality, increased number of channels and an interactive service branded Open.... now called Sky Active, BSkyB competed with the ONdigital (later ITV Digital) terrestrial offering and cable services. Within 30 days, over 100,000 digiboxes had been sold, which help bolstered BSkyB's decision to give away free digiboxes and minidishes from May 1999.

What happened first, the giveaway of digiboxes, or the launch of the Sky Digital brand?

What did digital offer over analogue?

Which had more channels, Sky's digital service, or analogue?

This combination of cancellations and σ and π overlaps results in dioxygen's double bond character and reactivity, and a triplet electronic ground state. An electron configuration with two unpaired electrons as found in dioxygen (see the filled π* orbitals in the diagram), orbitals that are of equal energy—i.e., degenerate—is a configuration termed a spin triplet state. Hence, the ground state of the O2 molecule is referred to as triplet oxygen.[b] The highest energy, partially filled orbitals are antibonding, and so their filling weakens the bond order from three to two. Because of its unpaired electrons, triplet oxygen reacts only slowly with most organic molecules, which have paired electron spins; this prevents spontaneous combustion.

What is the chemical name for oxygen?

Which orbitals are only partially filled in O2?

What does it mean if two orbitals are degenerate?

What is a spin triplet state?

What are in the orbitals of atoms?

A constitutional change was considered that would eliminate the position of Prime Minister and simultaneously reduce the powers of the President. A referendum to vote on the proposed constitution was held on 4 August 2010, and the new constitution passed by a wide margin. Among other things, the new constitution delegates more power to local governments and gives Kenyans a bill of rights. It was promulgated on 27 August 2010 at a euphoric ceremony in Nairobi's Uhuru Park, accompanied by a 21-gun salute. The event was attended by various African leaders and praised by the international community. As of that day, the new constitution heralding the Second Republic came into force.

What was the main focus, in relation to the head of cabinet of Kenya of a radical divergence from the previous set of guidelines for a country that was bounced off the legislators in Kenya?

What in the radical divergence from the previous set of guidelines that was instituted in the summer of 2010 seeks to redress injustices?


D(RoBERTa)

Between 1991 and 2000, the total area of forest lost in the Amazon rose from 415,000 to 587,000 square kilometres (160,000 to 227,000 sq mi), with most of the lost forest becoming pasture for cattle. Seventy percent of formerly forested land in the Amazon, and 91% of land deforested since 1970, is used for livestock pasture. Currently, Brazil is the second-largest global producer of soybeans after the United States. New research however, conducted by Leydimere Oliveira et al., has shown that the more rainforest is logged in the Amazon, the less precipitation reaches the area and so the lower the yield per hectare becomes. So despite the popular perception, there has been no economical advantage for Brazil from logging rainforest zones and converting these to pastoral fields.

What consumer product are the rain forests cleared for?

Of the two countries that produce soybeans, which country is clearing rain forest in order to increase production?

How is the rain forest cleared?

Tamara de Lempicka was a famous artist born in Warsaw. She was born Maria Górska in Warsaw to wealthy parents and in 1916 married a Polish lawyer Tadeusz Łempicki. Better than anyone else she represented the Art Deco style in painting and ar. Nathan Alterman, the Israeli poet, was born in Warsaw, as was Moshe Vilenski, the Israeli composer, lyricist, and pianist, who studied music at the Warsaw Conservatory. Warsaw was the beloved city of Isaac Bashevis Singer, which he described in many of his novels: Warsaw has just now been destroyed. No one will ever see the Warsaw I knew. Let me just write about it. Let this Warsaw not disappear forever, he commented.

What is the pseudonym of the artist born to wealthy parents in Warsaw?

Into what family did the artist who represented the Art Deco style marry?

What was the birthplace of the composer who attended the Warsaw Conservatory?

Among the most important classes of organic compounds that contain oxygen are (where "R" is an organic group): alcohols (R-OH); ethers (R-O-R); ketones (R-CO-R); aldehydes (R-CO-H); carboxylic acids (R-COOH); esters (R-COO-R); acid anhydrides (R-CO-O-CO-R); and amides (R-C(O)-NR2). There are many important organic solvents that contain oxygen, including: acetone, methanol, ethanol, isopropanol, furan, THF, diethyl ether, dioxane, ethyl acetate, DMF, DMSO, acetic acid, and formic acid. Acetone ((CH3)2CO) and phenol (C6H5OH) are used as feeder materials in the synthesis of many different substances. Other important organic compounds that contain oxygen are: glycerol, formaldehyde, glutaraldehyde, citric acid, acetic anhydride, and acetamide. Epoxides are ethers in which the oxygen atom is part of a ring of three atoms.

In addition to having oxygen, what do alcohols, ethers and esters have in common, according to the article?

What letter does the abbreviation for acid anhydrides both begin and end in?

Which of the organic compounds, in the article, contains nitrogen?

Which of the important classes of organic compounds, in the article, has a number in its abbreviation?