By Elizabeth M. Sacksteder and Ross M. Gotler, Paul, Weiss, Rifkind, Wharton & Garrison LLP. Ms. Sacksteder is a Partner with the firm and a member of the Commercial Division Advisory Council. Mr. Gotler is E-Discovery Counsel with the firm. Some of this material first appeared in Law 360.
The most expensive stage of big-ticket litigation today is review of the huge volume of electronically-stored information (ESI) that such cases typically require, notwithstanding such common economies as the use of vendors to do first-level document review. Achieving greater efficiency in this resource-intensive stage of litigation—making review of ESI cheaper, faster, and more accurate—is a shared goal of litigants, their counsel, and the courts. Sophisticated litigants know that the use of technology-assisted review can yield substantial cost savings as well as streamline and accelerate document review and production.
Though the e-discovery industry is embracing technology, neither the Federal Rules of Civil Procedure nor state procedure codes address whether, in what circumstances, or how a party may use technology-assisted review to fulfill its disclosure obligations. Other than references in a few discovery pilot programs, a relatively sparse body of mostly federal case law, and secondary sources such as the commentaries of The Sedona Conference, there has been little express guidance to date for practitioners or courts concerning the appropriate use of technology-assisted review.
A new rule applicable in the Commercial Division of the New York State Supreme Court, which becomes effective today, October 1, 2018, aims to fill that gap. We examine this first-of-its-kind discovery rule and make the case that other courts should give it a close look and assess whether such a rule would work in their jurisdictions.
The New Commercial Division Rule
The new rule—codified as Rule 11-e(f) of the Rules of Practice for the Commercial Division—provides:
The parties are encouraged to use the most efficient means to review documents, including electronically stored information (‘ESI’), that is consistent with the parties’ disclosure obligations under Article 31 of the CPLR and proportional to the needs of the case. Such means may include technology-assisted review, including predictive coding, in appropriate cases. The parties are encouraged to confer, at the outset of discovery and as needed throughout the discovery period, about technology-assisted review mechanisms they intend to use in document review and production.
The principal import of the new rule is three-fold. First, the rule confirms that efficiency is not only an appropriate consideration in determining how to review ESI, but that the most efficient means to review documents is actually preferred, provided that the method chosen by the producing party meets that party’s disclosure obligations. Second, the rule clarifies that the use of technology-assisted review, including predictive coding and other algorithmic, machine-learning-based technologies, is encouraged when those methods offer the most efficient means for a party to meet its disclosure obligations. And, third, the rule reinforces the expectation that, as with other aspects of the discovery process, parties are expected to confer with one another about the technology-assisted mechanisms, if any, they intend to use in document review and production.
The new rule does not, however, prescribe or limit which technologies may be used. That is a sensible approach for several reasons.
First, these technologies are changing rapidly. Any court rule that purported to define which technology-assisted review techniques are permissible would quickly become outdated.
Second, what is appropriate for one case may not be appropriate for another case. One of the following options may be cost-effective to help cull a data set and organize it for human review, even if the volume of documents collected is not overwhelming:
- software-based technologies such as keyword searching, which identifies words or phrases that are likely to be found in responsive documents;
- concept searching, which identifies documents incorporating concepts similar to the specific search terms used;
- email threading, which packages together email strings and any attachments as one chronological thread; near-duplicate identification, which groups together similar documents;
- clustering, which uses conceptual analytics technology to group and categorize similar documents.
Techniques based on a machine-learning algorithm, such as predictive coding, however, may require an upfront investment of time in “teaching” the computer to recognize the characteristics of responsive documents, often iteratively, and thus may not be cost-effective unless the volume of ESI to be reviewed is sufficiently large.
Third, the Commercial Division rule appropriately charges the producing party with choosing an approach to ESI review that is compliant with that party’s discovery obligations. The rule offers no safe harbor. The producing party thus has both the right and the responsibility to choose, from among the tools available to it, the document review approach (whether technology-based or not) that it deems most efficient and consistent with its discovery obligations, in light of the volume and characteristics of the data set it has collected for review and the document requests to which it is responding.
At the same time, the rule encourages parties to confer up front about the document review mechanisms they have chosen, so that any concern about the sufficiency of the review can be identified and resolved early.
Is a Court Rule on Technology-Assisted Review a Good Idea?
Nothing about the Commercial Division’s new rule is out of step with the approach other courts have taken when parties have asked them to consider the appropriateness of particular methods of technology-assisted review in particular cases. In the Southern District of New York, for example, Magistrate Judge Andrew Peck (ret.) has observed that “computer-assisted review is an available tool and should be seriously considered for use in large-data-volume cases.” He expressly declined to endorse any particular tool, vendor, or ESI protocol, or to mandate computer-assisted review in all cases. Da Silva Moore v. Publicis Groupe, 287 F.R.D. 182, 193 (S.D.N.Y. 2012).
The Delaware Chancery Court has required a party to supplement its production using predictive coding, rather than the manual review its adversary requested, and to work with its adversary and an agreed vendor “to formulate the procedure to identify documents responsive to Defendants’ document requests using the predictive coding process.” OSI Restaurant Partners, LLC v. United Ohana, LLC, 2017 WL 396357 (Del. Ch. Ct. 2017).
Foreign courts have likewise recognized the utility of predictive coding in reviewing large volumes of ESI. See, e.g., Irish Bank Resolution Corp. Ltd & ors v. Quinn & ors,  IEHC 175 (Ir.); Brown v. BCA Trading Ltd.,  EWHC 1464 (Ch) (Eng.); Pyrrho Invs. Ltd. v. MWB Prop. Ltd.,  EWHC 256 (Ch) (Eng.).
Only in extraordinary situations have courts denied a party’s request to use predictive coding, such as where allowing its use would have meant amending the existing discovery order over objection in favor of a producing party that had failed to confer with its adversary or the court about its intended use of predictive coding. See, e.g., Progressive Casualty Ins. v. Delaney, 2014 WL 3563467 (D. Nev. 2014).
Why, then, have a procedural rule, instead of having courts address the use of technology-assisted review in the context of particular cases? A rule cannot address every situation or eliminate all disputes. The principal benefit of a rule of general application, however, is that it provides litigants with assurance from the commencement of an action that technology-assisted review is in bounds. Litigants cannot legitimately object to a producing party’s use of technology-assisted review. Such a rule recognizes the reality that, as society becomes ever more reliant on electronic communications of all kinds, and as advanced technologies such as machine learning and artificial intelligence grow ever more sophisticated, technology-assisted review will become an increasingly important tool in litigators’ toolkits.
Similarly, a rule of general application provides guidance to judges, some of whom may have practiced law only in the era before large volumes of ESI became the norm in high-stakes litigation, that the use of machines is not only routine in discovery practice today but a development to be applauded and encouraged. At the same time, by codifying the expectation that parties will confer about any use of technology-assisted review, such a rule (consistent with the preliminary conference and meet-and-confer requirements of most federal and state courts) will, early in the discovery process, bring to the surface disagreements and promote their consensual resolution before a party has invested time and money on a review technique to which the other party may object.
Commentators have generally lauded the Commercial Division’s adoption of the rule. A New York Law Journal article, for example, quoted a practitioner’s description of the new rule as “revolutionary.” A National Law Review piece asserted that “[t]his explicit encouragement of the use of technology-assisted review in the court rules  places the Commercial Division at the forefront of complex litigation courts.”
That positive reaction is not surprising. Discovery squabbles are the part of litigation practice that practitioners, and courts, typically dislike the most. And, as one federal judge recently noted, “[d]iscovery of ESI is still discovery, governed by the same [rules] as all other civil discovery.” City of Rockford v. Mallinckrodt ARD Inc., 2018 WL 3766673, *2 n.2 (N.D. Ill. Aug. 7, 2018). A court rule confirming that a producing party has discretion to select any means to review its documents that is reasonably calculated to comply with the applicable discovery rules and scheduling orders, whether technology-assisted or not, can only serve to reduce such squabbles.
By expressly approving technology as a tool to be embraced where it can improve the speed and quality of the discovery process, rules like that adopted in the New York Commercial Division can help to put the focus of the discovery process where it should be: on the substance of the discovery, not the mechanics.