10/26/2015 – The Black Box Society chapters 4-6

This week we conclude our discussion of The Black Box Society.

NOTE: The reading is a little longer than usual this week, so you should allow extra time to finish it.

Please be ready to state which book you are reading for the student presentations at the end of the quarter.

Comment lengths are restricted to 300 words or less.



Filed under Class sessions

14 responses to “10/26/2015 – The Black Box Society chapters 4-6

  1. Throughout the book, Pasquale draws comparisons between authoritarian states and powerful corporations like internet monopolies and finance giants. The “lords of the cloud” (145) have ultimate power, and “no one has a right to question the unilateral decision of the ruler” (163). Internet platforms like Google and Facebook have “populations rivaling those of small countries” (198)–actually they would be the largest countries in the world–and have “essentially statelike characteristics” (198).

    I agree wholeheartedly with this characterization of private power, which is why most of Pasquale’s proposals seem weak to me, especially when it comes to tech. In Chapter 5, he argues that “it’s not enough just to watch the key firms controlling our information, our media…We have to be able to improve them” (140). He suggests doing so by stricter enforcement of existing policies (145), enhancing the rights of consumers to “inspect, correct, and dispute inaccurate data” (145), applying competition law to search engines (161), and several other ways of emboldening regulatory agencies.

    In the final chapter Pasquale goes a bit further, saying “[if] we were to promote [industry-controlled] public functions directly, we might begin to see some real accountability” (208). He suggests a “public ordering” (209) of search results and some sort of public financing of online art, like Dean Baker’s “artistic freedom vouchers” (202).

    Proposals involving public control like a publicly oriented search algorithm and democratically supported artists hit the heart of the problem in a way that regulatory constraints can’t, by aligning incentives rather than balancing them. Internet monopolies, with built-in interactivity, clean user interfaces, and relatively simple business models, are some of the best candidates for cooperative worker and consumer control, and we should think creatively about solutions that avoid the risks of (corporate-)state and private power while approaching something like real democracy.

  2. Pasquale presents two interesting rhetorical questions near the end of the book to compare the financial sector to the reputation and search sectors: “What is money (and all its derivatives) other than /information/ about how much of our collective goods and services its owners can demand? And what are reputation and search firms establishing other than new /currencies/ for allocating opportunity and attention?” (215). The uniting factor, Pasquale says, is the processing of information. I like the sound of this metaphor, but I think it may be somewhat contentious.

    For example, it’s fairly established that the purposes of money are to store value, and to act as a unit of account and exchange. That is, the best money will serve these three functions well. I’m not certain if the technical difference between money and capital (if there is one) comes into play here, but if this is what the financial sector moves and manipulates, then it seems unwarranted to make the given comparison to the search and reputation sectors. Obviously they deal with processing information, and that information has value insofar as it represents social capital (if we’re talking about the reputations of people and businesses). But this information is much more functionally limited than money. It doesn’t act as a unit of account or exchange, for instance.

    Thus I think the financial sector has a significantly higher burden for processing its information ethically, and Pasquale’s claim here is much less warranted than his metaphor might suggest. I absolutely agree that the search and reputation sectors have an unmet responsibility, but the information they deal with is, in my mind, incomparable to that of the financial sector, and I think this is important when comparing the importance of transparency between these sectors.

  3. After showing the dangers to our society that black box industries can create in the first few chapters, Pasquale aims to conclude the book with a solution to prevent the prophecy given by the black box industries and perpetuated by modern day society. Because of the complexities of the black box industries, it is clear that the solution must be multi-faceted with various actions and regulations of sorts to deal with the different issues. There are not surprisingly challenges that arise surrounding proposed solutions. However, by providing examples of ways that different proposed parts of the solution have been successful on a smaller scale, Pasquale paints an optimistic picture for turning around the future of our black box society. For example, technology firms are touted as glamorous companies, and it is just naturally difficult for the government to keep up with the fast paced nature of these technology firms (198). Pasquale’s smaller scale example to show that this is not necessarily an impossible problem to solve is that other countries in Europe are requiring Google to alter certain search results that defame individuals or mislead users (198). A proposed solution to banking scandals is the “social movement to “Move Your Money” out of the big banks… a valuable low-cost option to the millions of ‘unbanked’ Americans” (209). Pasquale follows up by saying, “this is not a radical idea: the Bank of North Dakota has offered the state’s farms and businesses loans for almost a century.” (210). While Pasquale acknowledges that on a large scale this seems like a “radical” and hard-to-execute solution, by providing an example of a successful practice of it on a smaller scale gives the feeling that there is hope for eventually implementing a nation wide solution in a similar form.

  4. I read the last 75 pages of his book as a repeated call to regulate and audit these black box institutions and to empower what are now feeble governmental institutions. And as I read the final chapters I was waiting for the section where he would rip apart lobbyists, campaign financing and Super PACs, and the vast systemic control that the private sector has on the government. Finally, the section came; however, it was only a page and a half.

    As someone who read Pasquale’s book and was extremely interested and convinced by what he had to say I was disappointed he did not explore the issue of systemized political corruption as one of the primary catalysts for the lack of regulation of black boxes. On page 206, he introduces a section titled On the Narrowing Divide between Government and Business where he quickly touches on a few brief points such as the fact that Google’s corporate lobbying expenditures are the second highest in the country. As a reader, I felt like I wanted to hear more about this topic as I believe, after reading the book, that it is the primary reason black boxes exist so pervasively. In my eyes, Pasquale failed to deliver on this topic which I think would have fueled his argument even further by soliciting the reader’s anger regarding the topic of what is essentially “legalized” corruption. One route he could have potentially taken, instead of only focusing on repeated calls for more regulation, would be a discussion on how to weed out this influence of the private sector (ie. overturning Citizens United, perhaps). I think he missed an opportunity to further his argument by not spending more time on this topic as it is a necessary step in the path for more regulation.

  5. acamperi

    In his chapter about watching the watchers, Pasquale mentions a couple of potential solutions to the issue of misuse of information or funds, but also a couple of potential drawbacks to these solutions, and I would like to discuss these drawbacks and talk about why oversight is, in my opinion, not entirely feasible.

    Pasquale first talks about how the algorithms that these people use should be publicly available for inspection, or at least inspected by a “trusted auditor” (141). However, this strikes me as a not very satisfactory solution. First of all, the number of people that could actually understand these algorithms and be able to tell if there is some fishy business going in is extremely low. These would be very sophisticated pieces of software that only a select few people would be able to understand, and so the rest of the public would have to rely on these people, these trusted auditors, for information. However, this becomes the same problem as before: who is going to watch these auditors to make sure that they themselves are not gaming the system for their own personal gain. One solution to this that Pasquale discusses is the possibility of publicly available audit logs for these inspections, so that any citizen could have access to the records (within reason), and therefore be able to track any misuse of information or funds (157-158). However, even he acknowledges that, practically, these logs would contain too much information for people to be able to thoroughly inspect them, which defeats the purpose.

    While I do agree with many of the points that Pasquale makes, I still believe that where there is power, as there is here, there will be abuse of power, and there is no practical way of preventing that.

  6. ccibils

    Pasquale does a remarkable job at illustrating how obfuscated and obscure the financial system has become. There is no question that deception and information asymmetry are core the trade, and his views on how the big Wall Street banks operate are very clearly grim. I stand with him completely, and the final taste the book left was that it is a strong reaction from a lawman to the events of 2008. He is not wrong in pointing out that systemic flaws run contrary to the interests of the average citizen, and in favor of the interests of massive corporations. However, I don’t think his proposals his proposals for solutions are feasible, even though I think they would work.
    His solutions sound a bit too optimistic and utopian, as if there’s an easy way for the citizenry to request that such agencies be created, such regulation imposed, and such moral questions answered. Given the current political climate and system, I believe it would take a whole candidacy, much like Larry Lessig’s, to trump the current system. Maybe even that would not be enough! Why would Congress bite the hand that feeds? The system is broken and sadly the issues in the book are just a symptom of a deeper disease.
    However, that by no means that I didn’t find value in his proposals. Quite the contrary, I’m extremely happy I got to read a critique so well written about the system as it is today, and thought out solutions to how to cure these symptoms. Material like this, strong opinions about corrupt systems, should be more generally available, and I believe that it is a great thing for these comments to be voiced. It is hard to finish reading the book without feeling sad and small, but without this knowledge there is really no way for society to move forward in a fair and just way.

  7. I’m pretty compelled by Pasquale’s overall argument about the inherent problem of black boxes. However, I wonder if more of his examples could have also been directed at the black box of policy-making. While he seemingly alluded to this concept often when he spoke of lobbyists, the secretive operation of government regulators, and loopholes in Dodd-Frank, it seems that Pasquale missed an opportunity to discuss the political process as a black box of similar caliber to the others.

    Part of the reason that lobbying has become so effective today is that the passage of legislation is sufficiently obfuscated from the public eye. Very few people in America know who their congressman/women is receiving campaign donations from, nor do they take the time to reach out to him or her to express their own opinion. While Congressional testimony is open and on the record, almost every bill these days is too long to read (it’s not just Dodd-Frank), and many are passed only after closed-door deals.

    None of this is all that new, but I think it poses the question of whether a black box is capable of dismantling another black box. It seems that both boxes profit off each other, and so it’s hard to know what would create enough incentive for the government to start implementing the solutions that Pasquale proposes (i.e. regulatory changes (209) or the creation of more public services (205)). And as long as the overall political process is hidden, there will likely be little pressure for specific regulators to change their methodology. Therefore, as much of a problem as I think the black boxes of finance and search are, it seems like there is quite another black box that we may have to fully acknowledge first just to give us the tools to address them.

  8. sgussman

    Pasquale discusses two potential legal challenges to Google from pages 165 to 168. The first revolves around antitrust law and the First Amendment, while the second is more akin to malpractice suits.

    In the past Google claimed that its search results are speech, and thus protected under the 1st amendment. Pasquale, however, proposes a precedent in which First Amendment did not shield a company’s anti-competitive practices. In the Lorain Journal v. United States, the Lorain Journal stopped running ads for companies that also advertised with their competitor, and the Supreme Court ruled that the journal’s actions wasn’t protected free speech because it was motivated by economic means and therefore they were in violation of the Sherman Act.

    Google, however, claims that the Lorain precedent does not apply because Google is motivated by purely editorial reasons, whereas the Lorain Journal was motivated by economic reasons. This is, however, disputed by many competitors to Google and is impossible for Google to prove unless it releases its search-ranking algorithms (which it won’t).

    This appears to be a strong legal challenge capable of forcing Google to be open about it search rankings, but it may have been weakened by Google’s recent restructuring. Now that Google and it’s acquisitions are being broken into separate companies under the holding company Alphabet. By splitting the acquisitions from the search engine, it will be more difficult in the future to prove that Google is engaging in anticompetitive practices. When Google was unified (with interconnected finances), it would profit directly if, say, a competitor to Nest (which is an acquisition/subsidiary) dropped in search rankings. Now, however, it’s much more difficult to prove that Google benefits in said situation because it no longer receives a direct financial incentive to change the search results. Google can thus claim that it’s acting editorially and the Lorain precedent doesn’t apply because it has no direct-economic motivations.

  9. aselvan2012

    Chapter 4 of The Black Box Society was something I had been looking forward to reading for a while. As someone who has worked in a variety of different realms within finance, it is true that the entire system is a bit of a black box. However, I worry that in his explanation, Pasquale is obfuscating the reader. He mentions high frequency trading, quantitative easing, futures trading and bad accounting (mark to model instead of mark to market) all within a few pages, leading readers to believe the entire financial system is a bad thing. Each of those aforementioned topics could have entire volumes written about them. As someone who worked in futures trading, I completely disagree with his statement that short-term horizons steer money away from things like green energy (pg 128). In fact, it is actually thanks to complex derivative structures that financing is able to be guaranteed for risky projects such as wind farms. On a broader theme, futures are some of the core tools that enable companies to reduce their risk (hence the 13 bankers in Summers’ office), meaning cheaper airlines tickets or car frames – and I see no one complaining about that.
    I worry that a book like this, though correctly insightful in many aspects, has the power to influence people, either voters, or politicians crafting legislation to come to wrong conclusions about what is a highly complex field. I worry that even in this class, standing up against some of the things Pasquale implies casts the image upon me as one of the people siding with millionaire bankers. Whilst I do believe more people need to be asking questions about what goes on in our financial system, a strong insight is needed into how it really works.

    Happy to discuss more nuances about what Pasquale has said in class or after

  10. Betsy Alegria

    In Chapter 5, Pasquale outlines a list of promises Facebook failed to keep to its users (145). Surprisingly, Facebook received very little monetary fines and it’s unclear if anyone was fired or punished. When I read this, I thought about two things – 1) I am an active user of Facebook and I didn’t even know that that had happened and 2) I wondered if knowing this information would affect how I used Facebook. To be honest, I don’t think it would, unless if other people on Facebook stopped using it because of this reason. From the population of Facebook users, how many would change how they use the site knowing this information?

    In the same chapter, Pasquale talks about how people mechanically click “I agree” when confronted with “terms of service” agreements, because they are fully aware that there’s no chance they can modify those terms (144). This speaks to the power structure that has been set up between us and these larger companies we use so often but don’t even know how they operate. I am someone who mechanically clicks “I agree,” and now I wonder why it is that I don’t read it and see what I really am agreeing to. Pasquale is right when he says that on the other hand, trying to make us engage would be too much of an effort.

    With both of the above examples, I wondered if the same was true to those who are/are not digital natives. Secondly, I wonder what psychological concepts (ie. groupthink) are so strong here that we, the users, don’t feel a sense of urgency to protect our privacy/rights online or investigate the mysteries of the black boxes. With any structural system, perhaps it is inflicting those with the least power/stake to change it.

  11. Jamison Elizabeth Searles

    Throughout the book, Pasquale raises concerns about the way big companies are handling and representing data about their users. One practice I wanted to discuss is doxing, which is the act of gathering information on an individual via the Internet and publishing that information usually with the intent of connecting defaming material with a user. Often, “doxers” will release the address, real name, or occupation of an individual they believe deserves punishment. For the most part, these practices are legal, so long as the information was gained through legitimate means, and many times, the information used to ruin a user’s name was published online by the user himself. Victims of doxing have included Reddit “trolls” and KKK members. Many doxed individuals lose their jobs, families, and always, their reputation. While these groups don’t usually represent society’s most upstanding members, do they still have a right to privacy and a right to recover from past mistakes? Furthermore, doxers often misidentify their targets, resulting in irreparable damage to innocent names.

    My question is, while search engines have a responsibility to handle users’ data respectfully on their end, what role should they play in the regulation of information sharing by users intended to harm other users, if any at all? Pasquale endorses the steps taken by Germany, Argentina, and Japan to require Google to protect users against search results that defame them (198), but when the users themselves shared this information and it is accurate, do these platforms have any duty to prevent them from being doxed?

  12. Chapter 5 of The Black Box Society talks of the informational asymmetries inherent cemented into the fabric of Wall Street and Silicon Valley. Pasquale argues that regulators are woefully underfunded (in the case of Wall Street) or just nonexistent (in the case of the Valley’s data glut) to be able to perform their jobs competently. I agree that competent regulation may be necessary to protect citizens from corporate interests, but think that creating an “FDA for finance” [p190], such as the one proposed by “two scholars of law and economics”, seems to misunderstand the nature of the problem and present thorny issues surrounding the way that the private sector has its innovation regulated by the state.

    There are several points Pasquale fails to consider about the “FDA for finance” proposal, of which we will consider one – the nature of the industry being regulated.

    The goods regulated by the FDA are goods we ingest every day, and ‘mistakes’ in those products have and will kill thousands of people every year. The safety of food and drugs is directly correlated to the physical health of humans. The safety of financial derivatives is not. It is a well-functioning market, not the state, that determines the risk of financial products, and therefore their viability.

    If the “FDA for finance” were to be a regulatory body charged with approving financial products before they were allowed to hit the open market, all it would be doing is preventing market players with insufficient intellectual capital to properly price the risks of those products to have others rate it for them. This task is one already completed by private companies, and it seems inefficient for the state to duplicate their work. (Perhaps, however, there should be stricter guidance as to how such companies handle conflicts of interest.)

  13. Jack Cook

    I really enjoyed Pasquale’s coverage of technology in the banking sector, although I disagree with some of the conclusions he reaches. Maybe ‘disagree’ is too strong a word – the core of the problem as he presents it is the idea of a “black box” algorithm, an algorithm that makes influential decisions about us but whose inner workings are unknown to us and are therefore immune from scrutiny. This thesis presents what is happening in the tech and financial sectors as a temporally unique consequence of the prevalence of technology in our society.

    After this week’s readings I can’t help but feel that his thesis moves increasingly away from this initial point and instead draws us to the conclusion that massive deregulation/slow regulation is responsible for the problems in the tech and finance industries. Had I stopped reading at Chapter 2 I would have left thinking that proprietary algorithms themselves were to blame for unethical practices. Now I am more confident than ever that these black box algorithms are a symptom of deregulation and not a protection from it. As a weak form of evidence, I submit that there were some economists who identified the housing bubble and danger of CDO and property asset derivative trading well before the 2008 economic crisis (1). Even though Pasquale’s remedial steps for increased consumer protection in Chapter 6 include a lot of regulatory suggestions, I can’t hep but feel that his obsession with the almighty “black box” detracts from his stronger argument that Wall Street and Silicon Valley have only the most circumspect regulatory oversight. It’s also hard to look at the evidence he presents in the book about the close relationship enjoyed between these industries and government and not be drawn to the conclusion that even the blackest box could hide practices that regulators routinely ignore.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s