Premiered Dec 6, 2020 Alvin Ailey American Dance Theater is celebrating six decades of “Revelations,” a choreographic masterpiece that has become a lasting cultural treasure beloved by generations. Watch the company’s full performance from the 2015 Lincoln Center at the Movies production! The piece, which premiered in 1960, pays homage to and reflects African-American cultural heritage, which Ailey considered one of America’s richest treasures – “sometimes sorrowful, sometimes jubilant, but always hopeful.” To continue celebrating the anniversary of this iconic work, check out Alvin Ailey American Dance Theater’s digital season, which will elevate Ailey’s legacy of innovation and excellence with premieres, interactive workshops, artist conversations and more. Visit AlvinAiley.org for details!
There’s a passage in Ernest Hemingway’s novel The Sun Also Rises in which a character named Mike is asked how he went bankrupt. “Two ways,” he answers. “Gradually, then suddenly.”
Ernest Miller Hemingway (July 21, 1899 – July 2, 1961) was an American novelist, short-story writer, and journalist. His economical and understated style—which included his iceberg theory—had a strong influence on 20th-century fiction, while his adventurous lifestyle and public image brought him admiration from later generations. Wikipedia
“There are decades where nothing happens; and there are weeks where decades happen.”
― Vladimir Ilyich Lenin
Vladimir Ilyich Ulyanov, better known as Vladimir Lenin (April 22, 1870 – January 12, 1924), was a Russian revolutionary, Soviet politician, and political theorist who served as the first and founding head of government of Soviet Russia from 1917 to 1924 and of the Soviet Union from 1922 to 1924. Wikipedia
Content owners are wising up to their work being freely used by Big Tech to build new AI tools.
Bots like Common Crawl are scraping and storing billions of pages of content for AI training.
With less incentive to share online freely, the web could become a series of paywalled gardens.
AI is undermining the web’s grand bargain, and a decades-old handshake agreement is the only thing standing in the way.
A single bit of code, robots.txt, was proposed in the late 1990’s as a way for websites to tell bot crawlers they don’t want their data scraped and collected. It was widely accepted as one of the unofficial rules supporting the web.
At the time, the main purpose of these crawlers was to index information so results in search engines would improve. Google, Microsoft’s Bing and other search engines have crawlers. They index content so it can be later served up as links to billions of potential consumers. This is the essential deal that created the flourishing web we know today: Creators share abundant information and exchange ideas online freely because they know consumers will visit and either see an ad, subscribe, or buy something.
Now, though, generative AI and large language models are changing the mission of web crawlers radically and rapidly. Instead of working to support content creators, these tools have been turned against them.
The bots feeding Big Tech
Web crawlers now collect online information to feed into giant datasets that are used for free by wealthy tech companies to develop AI models. CCBot feeds Common Crawl, one of the biggest AI datasets. GPTbot feeds data to OpenAI, the company behind ChatGPT and GPT-4, currently the most powerful AI model. Google just calls its LLM training data “Infiniset,” without mentioning where the vast majority of the data comes from. Although 12.5% comes from C4, a cleaned up version of Common Crawl.
The models use all this free information to learn how to answer user questions immediately. That’s a long way from indexing a web site so users can be sent through to the original work.
Without a supply of potential consumers, there’s little incentive for content creators to let web crawlers continue to suck up free data online. GPTbot is already being blocked by Amazon, Airbnb, Quora, and hundreds of other websites. Common Crawl’s CCBot is beginning to be blocked more, too.
‘A crude tool’
What hasn’t changed is how to block these crawlers. Implementing robots.txt on a web site, and excluding specific crawlers, is the only option. And it’s not very good.
“It’s a bit of a crude tool,” said Joost de Valk, a former WordPress executive, tech investor and founder of digital marketing firm Yoast. “It has no basis in law, and is basically maintained by Google, although they say they do that together with other search engines.”
It’s also open to manipulation, especially given the voracious appetite for quality AI data. The only thing a company like OpenAI has to change is the name of its bot crawler to bypass all the disallow rules people put in place using robots.txt, de Valk explained.
Because robots.txt is voluntary, web crawlers can also simply ignore the blocking instructions and siphon the information from a site anyway. Some crawlers, like that of Brave, a newer search engine, don’t bother disclosing the name of their crawler, making it impossible to block.
“Everything online is being sucked up into a vacuum for the models,” said Nick Vincent, a computer science professor who studies the relationship between human-generated data and AI. “There’s so much going on under the hood. In the next six months, we will look back and want to evaluate these models differently.”
AI bot backlash
De Valk warns that owners and creators of online content may already be too late in understanding the risks of allowing these bots to scoop up their data for free and use it indiscriminately to develop AI models.
“Right now, doing nothing means, ‘I’m ok with my content being in every AI and LLM in the world,’ de Valk said. “That’s just plain wrong. A better version of robots.txt could be created, but it’d be very weird if that was done by the search engines and the large AI parties themselves.”
Several major companies and websites have responded recently, with some starting to deploy robots.txt for the first time.
As of August 22, 70 of the 1,000 most-popular websites have used robots.txt to block GPTBot since OpenAI revealed the crawler about three weeks ago, according to Originality.ai, a company that checks content to see if it’s AI-generated or plagiarized.
The company also found that 62 of the 1,000 most popular websites are blocking Common Crawl’s CCBot, with an increasing number doing so only this year as awareness of data crawling for AI has grown.
Still, it is not enforceable. Any crawler could ignore a robots.txt file and collect every last bit of data it found on a webpage, with the owner of the page more than likely having no idea it even happened. Even if robots.txt had any basis in law, its original purpose has little to do with information on the internet being used to create AI models.
“Robots.txt is unlikely to be seen as a legal prohibition on use of data,” according to Jason Schultz, director of NYU’s Technology Law & Policy Clinic. “It was primarily meant to signal that one did not want one’s website to be indexed by search engines, not as a signal that one did not want one’s content used for machine learning and AI training.”
‘This is a minefield’
This activity has been going on for years. OpenAI revealed its first GPT model in 2018, having trained it on BookCorpus, a dataset of thousands of indie or self-published books. Common Crawl started in 2008 and its dataset became publicly available in 2011 through cloud storage provided by AWS.
Although GPTBot is now more widely blocked, Common Crawl is a larger threat to any business that is concerned about its data being used to train another company’s AI model. What Google did for internet search, Common Crawl is doing for AI.
“This is a minefield,” said Catherine Stihler, CEO of Creative Commons. “We updated our strategy only a few years ago, and now we’re in a different world.”
Creative Commons started in 2001 as a way for creators and owners to license works for use on the internet through an alternative to strict a copyright framework, known as “copyleft.” Creators and owners maintain their rights, while a Commons license let people access the content and create derivative works. Wikipedia operates through a Creative Commons license, as does Flickr, Stack Overflow and ProPublica, along with many other well-known websites.
Under it’s new five year strategy, which notes the “problematic use of open content” to train AI technologies, Creative Commons is looking to make the sharing of work online more “equitable,” through a “multifrontal, coordinated, broad-based approach that transcends copyright.”
The 160 billion-page gorilla
Common Crawl, via CCBot, holds what is perhaps the largest repository of data ever collected from the internet. Since 2011, it has crawled and saved information from 160 billion web pages and counting. Typically it crawls and saves around 3 billion web pages each month.
Its mission statement says the undertaking is an “open data” project aimed at allowing anyone to “indulge their curiosities, analyze the world, and pursue brilliant ideas.”
The reality has become very different today. The massive amount of data it holds and continues to collect is being used by some of the world’s largest corporations to create mostly proprietary models. If a big tech company isn’t already making money off of its AI output (OpenAI has many paid services), there’s a plan to do so in the future.
Some big tech companies have stopped disclosing where they get this data. However, Common Crawl has been and continues to be used to develop many powerful AI models. It helped Google create Bard. It helped Meta train Llama. It helped OpenAI build ChatGPT.
Common Crawl also feeds The Pile, which hosts more curated datasets pulled from the work of other bot crawlers. It has been used extensively on AI projects, including Llama and an LLM from Microsoft and Nvidia, called MT-NLG.
One of The Pile’s most recent downloads from June is a massive collection of comic books, including the entire works of Archie, Batman, X-Men, Star Wars and Superman. Created by DC Comics, now owned by Warner Brothers, and Marvel, now owned by Disney, all of the works remain under copyright. The Pile also hosts a large set of copyrighted books, as The Atlantic recently reported.
“There’s a difference between the intent of crawlers and how they are used,” said NYU’s Schultz. “It is very hard to police or insist that data be used in a particular way.”
As far as The Pile is concerned, while it admits its data is full of copyrighted material, it claimed in its founding technical paper that “there is little acknowledgment of the fact that the processing and distribution of data owned by others may also be a violation of copyright law.”
Beyond that, the group, part of EleutherAI, argued its use of the material is considered “transformative” under the fair use doctrine, despite the data sets holding relatively unaltered work. It also admitted that it needs to use full-length copyrighted content “in order to produce the best results” when training LLMs.
Such arguments of fair use by crawlers and AI projects are already being put to the test. Authors, visual artists and even source code developers are suing the likes of OpenAI, Microsoft and Meta because their original work has been used without their consent to train something they get no benefit from.
“There’s no universe where putting something on the internet grants free, unlimited, commercial use of someone’s labor w/o consent,” Steven Sinofsky, a former Microsoft executive who’s a partner at VC firm Andreessen Horowitz, wrote recently on X.
No resolution in sight
For the moment, there’s no clear resolution in sight.
“We are grappling with all of this now,” said Stihler, the CEO of Creative Commons. “There are so many issues that keep cropping up: compensation, consent, credit. What does all of that look like with AI? I don’t have an answer.”
De Valk said Creative Commons, with its method of facilitating broader copyright licenses that allow owned works to be used on the internet, has been suggested as a possible model for consent when it comes to AI model development.
Stihler is not so sure. When it comes to AI, perhaps there is no single solution. Licensing and copyright, even a more flexible Commons-style agreement, likely won’t work. How do you license the whole of the internet?
“Every lawyer that I speak to says a license is not going to solve the problem,” Stihler said.
She’s is talking about this regularly to stakeholders, from authors to executives of AI companies. Stihler met with representatives of OpenAI earlier this year and said the company is discussing how to “reward creators.”
Still, it’s unclear “what the commons really looks like in the age of AI,” she added.
‘If we’re not careful, we’ll end up closing the commons’
Considering just how much data web crawlers have already scraped and handed over to big tech companies, and how little power is in the hands of the creators of that content, the internet as we know it could change dramatically.
If posting information online means giving data for free to an AI model that will compete with you for users, then this activity may simply stop.
There are already signs of this: Fewer human software coders are visiting Q&A web site Stack Overflow to answer questions. Why? Because their previous work was used to train AI models that now answer many of these questions automatically.
Stihler said the future of all created work online could soon look like the current state of streaming, with content locked behind “Plus” subscription fiefdoms that get ever more costly.
“If we’re not careful, we’ll end up closing the commons,” Stihler said. “There’ll be more walled gardens, more things people can’t access. That is not a successful model for humanity’s future of knowledge and creativity.”
That is what the Nobel-winning founding father of quantum mechanics Erwin Schrödinger (August 12, 1887–January 4, 1961) addresses in some exquisite passages from My View of the World (public library) — the slender, daring deathbed book containing two long essays penned on either side of his Nobel Prize, thirty-five years apart yet united by the unbroken thread of his uncommon mind unafraid of its own capacity for feeling, that vital capacity for living fully into the grandest open questions of existence.
It is relatively easy to sweep away the whole of metaphysics, as Kant did. The slightest puff in its direction blows it away, and what was needed was not so much a powerful pair of lungs to provide the blast, as a powerful dose of courage to turn it against so timelessly venerable a house of cards.
But you must not think that what has then been achieved is the actual elimination of metaphysics from the empirical content of human knowledge. In fact, if we cut out all metaphysics it will be found to be vastly more difficult, indeed probably quite impossible, to give any intelligible account of even the most circumscribed area of specialisation within any specialised science you please. Metaphysics includes, amongst other things — to take just one quite crude example — the unquestioning acceptance of a more-than-physical — that is, transcendental — significance in a large number of thin sheets of wood-pulp covered with black marks such as are now before you… A real elimination of metaphysics means taking the soul out of both art and science, turning them into skeletons incapable of any further development.
Even as he made his reality-reconfiguring contributions to science and its search for fundamental truth, Schrödinger never relinquished his passionate curiosity about philosophy and the ongoing questions of meaning that kernel every truth in the flesh of consciousness. He was as drawn to Spinoza and Schopenhauer as he was to the ancient Eastern traditions, and especially in their untrammeled common ground of panpsychism — one of the oldest and most notoriously misunderstood theoretical models of consciousness in relation to the universe.
A century after the pioneering Canadian philosopher, psychiatrist, and nature-explorer Richard Maurice Bucke drew inspiration from Whitman to develop his theory of cosmic consciousness, and a century before the emerging science of counterfactuals threw its gauntlet at our fundamental assumptions about the nature of the universe, Schrödinger takes up the parallels between the discoveries of quantum physics and the core ideas of the Hindu philosophy of Vedanta. In what might best be described as an existentialist prose-poem, he invites you to imagine yourself seated on a mountain bench at sunset, beholding a transcendent display of nature:
Facing you, soaring up from the depths of the valley, is the mighty, glacier-tipped peak, its smooth snowfields and hard-edged rock-faces touched at this moment with soft rose-colour by the last rays of the departing sun, all marvellously sharp against the clear, pale, transparent blue of the sky.
According to our usual way of looking at it, everything that you are seeing has, apart from small changes, been there for thousands of years before you. After a while — not long — you will no longer exist, and the woods and rocks and sky will continue, unchanged, for thousands of years after you.
What is it that has called you so suddenly out of nothingness to enjoy for a brief while a spectacle which remains quite indifferent to you?
In a passage evocative of Whitman’s timeless lines from “Crossing Brooklyn Ferry” — “Others will enter the gates of the ferry and cross from shore to shore… Others will see the islands large and small… A hundred years hence, or ever so many hundred years hence, others will see them… I am with you, you men and women of a generation, or ever so many generations hence … Just as you feel when you look on the river and sky, so I felt… Just as any of you is one of a living crowd, I was one of a crowd… What is it then between us?” — Schrödinger answers:
The conditions for your existence are almost as old as the rocks. For thousands of years men have striven and suffered and begotten and women have brought forth in pain. A hundred years ago, perhaps, another man sat on this spot; like you he gazed with awe and yearning in his heart at the dying light of the glaciers. Like you he was begotten of man and born of woman. He felt pain and brief joy as you do. Was he someone else? Was it not you yourself? What is this Self of yours? What was the necessary condition for making the thing conceived this time into you, just you and not someone else? What clearly intelligible scientific meaning can this ‘someone else’ really have? If she who is now your mother had cohabited with someone else and had a son by him, and your father had done likewise, would you have come to be? Or were you living in them, and in your father’s father… thousands of years ago? And even if this is so, why are you not your brother, why is your brother not you, why are you not one of your distant cousins? What justifies you in obstinately discovering this difference — the difference between you and someone else — when objectively what is there is the same?
Inconceivable as it seems to ordinary reason, you — and all other conscious beings as such — are all in all. Hence this life of yours which you are living is not merely a piece of the entire existence, but is in a certain sense the whole.
Once we fathom this fundamental reality of interbeing, Schrödinger observes, it becomes impossible to wish anything for ourselves that we do not wish for everyone else or to harm anyone else without harming ourselves:
It is the vision of this truth (of which the individual is seldom conscious in his actions) which underlies all morally valuable activity.
A decade later, in a lovely testament to Schrödinger’s insistence on the indivisibility of science and art in addressing those grandest unanswered question, Iris Murdoch — one of the vastest minds and finest literary artists of her time, and of all time — captured this elemental truth in her case for art as “an occasion for unselfing,” observing:
The self, the place where we live, is a place of illusion. Goodness is connected with the attempt to see the unself… to pierce the veil of selfish consciousness and join the world as it really is.
AUGUST 29, 2023 AT 7:00 AM BY ROB BREZSNY (NewCity.com)
Photo: Abhishek Koli
ARIES (March 21-April 19): Climate change is dramatically altering the Earth. People born today will experience three times as many floods and droughts as someone born in 1960, as well as seven times more heat waves. In urgent efforts to find a cure, scientists are generating outlandish proposals: planting mechanical trees, creating undersea walls to protect melting glaciers from warm ocean water, dimming the sun with airborne calcium carbonate, and covering Arctic ice with a layer of glass. In this spirit, I encourage you to incite unruly and even unorthodox brainstorms to solve your personal dilemmas. Be wildly inventive and creative.
TAURUS (April 20-May 20): “When love is not madness, it is not love,” wrote Spanish author Pedro Calderon de la Barca. In my opinion, that’s naive, melodramatic nonsense! I will forgive him for his ignorance, since he worked as a soldier and celibate priest in the seventeenth century. The truth is that yes, love should have a touch of madness. But when it has more than a touch, it’s usually a fake kind of love: rooted in misunderstanding, immaturity, selfishness, and lack of emotional intelligence. In accordance with astrological factors, I assign you Tauruses to be dynamic practitioners of genuine togetherness in the coming months: with hints of madness and wildness, yes, but mostly big helpings of mutual respect, smart compassion, tender care, and a knack for dealing maturely with disagreements.
GEMINI (May 21-June 20): Gemini author Iain S. Thomas writes, “There are two things everyone has. One is The Great Sadness and the other is How Weird I Really Am. But only some of us are brave enough to talk about them.” The coming weeks will be a favorable time to ripen your relationship with these two things, Gemini. You will have the extra gravitas necessary to understand how vital they are to your full humanity. You can also express and discuss them in meaningful ways with the people you trust.
CANCER (June 21-July 22): A self-fulfilling prophecy happens when the expectations we embrace actually come to pass. We cling so devotedly to a belief about what will occur that we help generate its literal manifestation. This can be unfortunate if the anticipated outcome isn’t good for us. But it can be fortunate if the future we visualize upgrades our well-being. I invite you to ruminate on the negative and positive projections you’re now harboring. Then shed the former and reinforce the latter.
LEO (July 23-Aug. 22): The holy book of the Zoroastrian religion describes a mythical mountain, Hara Berezaiti. It’s the geographic center of the universe. The sun hides behind it at night. Stars and planets revolve around it. All the world’s waters originate at its peak. Hara Berezaiti is so luminous and holy that no darkness can survive there, nor can the false gods abide. I would love for you to have your own version of Hara Berezaiti, Leo: a shining source of beauty and strength in your inner landscape. I invite you to use your imagination to create this sanctuary within you. Picture yourself having exciting, healing adventures there. Give it a name you love. Call on its invigorating presence when you need a sacred boost.
VIRGO (Aug. 23-Sept. 22): Virgo journalist Anthony Loyd has spent a lot of time in war zones, so it’s no surprise he has bleak views about human nature. He makes the following assertion: “We think we have freedom of choice, but really most of our actions are puny meanderings in the prison yard built by history and early experience.” I agree that our conditioning and routines prevent us from being fully liberated. But most of us have some capacity for responding to the raw truth of the moment and are not utterly bound by the habits of the past. At our worst, we have twenty-percent access to freedom of choice. At our best, we have seventy-percent. I believe you will be near the seventy-percent levels in the coming weeks, dear Virgo.
LIBRA (Sept. 23-Oct. 22): Libra poet T. S. Eliot wrote the iconic narrative poem “The Wasteland.” One part of the story takes place in a bar near closing time. Several times, the bartender calls out, “Hurry up, please—it’s time.” He wants the customers to finish their drinks and leave for the night. Now imagine I’m that bartender standing near you. I’m telling you, “Hurry up, please—it’s time.” What I mean is that you are in the climactic phase of your astrological cycle. You need to finish this chapter of your life story so you can move on to the next one. “Hurry up, please—it’s time” means you have a sacred duty to resolve, as best you can, every lingering confusion and mystery.
SCORPIO (Oct. 23-Nov. 21): Addressing a lover, Scorpio poet Margaret Atwood says, “I would like to walk with you through that lucent wavering forest of bluegreen leaves with its watery sun and three moons, towards the cave where you must descend, towards your worst fear.” That is a bold declaration. Have you ever summoned such a deep devotion for a loved one? You will have more power and skill than usual to do that in the coming months. Whether you want to or not is a different question. But yes, you will be connected to dynamic magic that will make you a brave and valuable ally.
SAGITTARIUS (Nov. 22-Dec. 21): Sagittarian theologian N. T. Wright writes, “The great challenge to self-knowledge is blind attachment to our virtues. It is hard to criticize what we think are our virtues. Although the spirit languishes without ideals, idealism can be the greatest danger.” In my view, that statement formulates a central Sagittarian challenge. On the one hand, you need to cultivate high ideals if you want to be exquisitely yourself. On the other hand, you must ensure your high ideals don’t become weapons you use to manipulate and harass others. Author Howard Bloom adds more. “Watch out for the dark side of your own idealism and of your moral sense,” he writes. “Both come from our arsenal of natural instincts. And both easily degenerate into an excuse for attacks on others.” Now is a good time for you to ponder these issues.
CAPRICORN (Dec. 22-Jan. 19): Capricorn playwright and novelist Rose Franken said, “Anyone can be passionate, but it takes real lovers to be silly.” That’s interesting, because many traditional astrologers say that Capricorns are the least likely zodiac sign to be silly. Speaking from personal experience, though, I have known members of your tribe to be goofy, nutty, and silly when they feel comfortably in love. An old Capricorn girlfriend of mine delighted in playing and having wicked good fun. Wherever you rank in the annals of wacky Capricorns, I hope you will consider expressing these qualities in the coming weeks. Romance and intimacy will thrive if you do.
AQUARIUS (Jan. 20-Feb. 18): As I work on writing new books, I often draw on inspirations that flow through me as I take long hikes. The vigorous exercise shakes loose visions and ideas that are not accessible as I sit in front of my computer. Aquarian novelist Charles Dickens was an adherent of this approach. At night, he liked to walk around London for miles, marveling at the story ideas that welled up in him. I recommend our strategy to you in the coming weeks, Aquarius. As you move your body, key revelations and enriching emotions will well up in you.
PISCES (Feb. 19-March 20): The coming months will be an excellent time to build, discover and use metaphorical bridges. To get in the mood, brainstorm about every type of bridge you might need. How about a connecting link between your past and future? How about a nexus between a task you must do and a task you love to do? And maybe a conduit between two groups of allies that would then serve you even better than they already do? Your homework is to fantasize about three more exciting junctions, combinations or couplings.
Aug 31, 2023 Latest Shows As Hurricane Idalia left a wake of destruction Wednesday, President Joe Biden said, “I don’t think anybody can deny the impact of the climate crisis anymore.” Climate activist and scientist Peter Kalmus calls for Biden to declare a climate emergency in order to unleash the government’s ability to transition away from fossil fuels. “The public just doesn’t understand, in my opinion, what a deep emergency we are in,” says Kalmus. “This is the merest beginning of what we’re going to see in coming years.” Kalmus blasts the fossil fuel industry for manipulating politics through campaign contributions, and GOP presidential candidates for misleading the public about climate science. “As a parent, as a citizen and as a scientist, I find it appalling and disgusting,” declares Kalmus. “I can’t mince words anymore.” Transcript: https://www.democracynow.org/2023/8/3…
Wake Forest Universi Apr 2, 2022 The Elizabeth & Robert Strickland Speaker Series brings religion scholars and distinguished religious professionals to speak on campus. Lectures are free and open to the public. This spring we welcome Elaine Pagels in a lecture based on her book Why Religion: A Personal Story. The book serves as a personal memoir dealing with the loss of a young child followed by the loss of a spouse all while asking universal questions about suffering and belief in the 21st century. Dr. Pagels is also well positioned to address the role of a University Church, an oxymoron to many. Dr. Pagels lecture will help the Wake Forest community answer this critique.
The Lord of Success is a card full of the promise of bounty. When we have achieved a degree of inner confidence and self-belief, we release new streams of energy which create a powerful and rewarding reality around ourselves. New ideas are easy to implement. New projects are fruitful. We are energised and enthusiastic about the work we have in hand.
This level of productive harmony comes from a deep-rooted trust in the self. Once we simply allow our power to flow, we find ourselves capable of high levels of success and fulfilment. These things flow naturally as a reward for the hard work we have invested in ourselves.
When the card comes up in a reading to indicate everyday matters, it promises that projects currently in hand will be lucrative and abundant. We will do exactly what we had hoped we might – and probably receive even more than we had hoped for. Financial and material matters will be positive and prosperous, allowing us to gain a stable and comfortable position.
There is often, during a period like this, such good fortune that we end up with more than we actually need. If this happens to you, make sure that you continue to allow money (which is after all only energy) to keep flowing. Use the abundance that comes to you, and be generous with your bounty. Ensure that others benefit appropriately from your abundance. That’s the best way that you can thank the Universe for flowing with you.