A_AT0 New_AJ0 Quantity_NN1 of_PRF Interest_NN1 in_PRP Judicial_AJ0 Behavior_NN1 
In_PRP this_DT0 section_NN1 ,_PUN we_PNP introduce_VVB an_AT0 original_AJ0 corpus_NN1 of_PRF Supreme_AJ0 Court_NN1 oral_AJ0 argument_NN1 audio_AJ0 recordings_NN2 scraped_VVN from_PRP the_AT0 Oyez_NP0 Project_NP0 (_PUL Cornell_NP0 2015_CRD )_PUR and_CJC develop_VVB a_AT0 new_AJ0 quantity_NN1 of_PRF theoretical_AJ0 interest_NN1 :_PUN judicial_AJ0 skepticism_NN1 ._SENT We_PNP first_ORD describe_VVB the_AT0 data_NN0 ,_PUN then_AV0 illustrate_VVB the_AT0 concept_NN1 of_PRF skepticism_NN1 with_PRP a_AT0 detailed_AJ0 case_NN1 study_NN1 ._SENT Finally_AV0 ,_PUN we_PNP extensively_AV0 validate_VVB the_AT0 model_NN1 and_CJC compare_VVB it_PNP with_PRP related_AJ0 work_NN1 analyzing_VVG the_AT0 text_NN1 of_PRF oral_AJ0 arguments_NN2 alone_AV0 ._SENT Importantly_AV0 ,_PUN we_PNP demonstrate_VVB that_CJT this_DT0 substantive_AJ0 quantity_NN1 is_VBZ not_XX0 measurable_AJ0 with_PRP existing_AJ0 methods_NN2 ._SENT Text-based_AJ0 classifiers_NN2 are_VBB on_PRP par_NN1 with_PRP random_AJ0 guessing_NN1 (_PUL or_CJC worse_AJC ,_PUN depending_VVG on_PRP the_AT0 evaluation_NN1 metric_AJ0 )_PUR ,_PUN whereas_CJS MASS_NN1 performs_VVZ nearly_AV0 as_CJS well_AV0 as_CJS human_AJ0 coders_NN2 ._SENT A_AT0 full_AJ0 analysis_NN1 of_PRF deliberation_NN1 in_PRP oral_AJ0 arguments_NN2 —_PUN a_AT0 theoretical_AJ0 analysis_NN1 that_DT0 is_VBZ possible_AJ0 only_AV0 with_PRP a_AT0 model_NN1 for_PRP speech_NN1 flow_NN1 —_PUN is_VBZ reserved_VVN for_PRP "_PUQ Testing_VVG Theories_NN2 of_PRF Supreme_AJ0 Court_NN1 Deliberation_NN1 ._SENT "_PUQ 
Audio_AJ0 Data_NN0 from_PRP Supreme_AJ0 Court_NN1 Oral_AJ0 Arguments_NN2 
We_PNP limit_VVB our_DPS analysis_NN1 to_PRP the_AT0 natural_AJ0 court_NN1 that_CJT begins_VVZ with_PRP the_AT0 appointment_NN1 of_PRF Justice_NP0 Kagan_NP0 and_CJC concludes_VVZ with_PRP the_AT0 passing_NN1 of_PRF Justice_NP0 Scalia_NP0 ,_PUN so_AV0 that_CJT the_AT0 composition_NN1 of_PRF the_AT0 Court_NN1 remains_VVZ constant_AJ0 for_PRP the_AT0 entirety_NN1 of_PRF the_AT0 period_NN1 we_PNP analyze_VVB ._SENT The_AT0 Oyez_NP0 data_NN0 contains_VVZ an_AT0 accompanying_AJ0 textual_AJ0 transcript_NN1 ,_PUN speaker_NN1 names_NN2 for_PRP each_DT0 utterance_NN1 ,_PUN and_CJC timestamps_NN2 for_PRP utterance_NN1 start_NN1 and_CJC stop_NN1 times_NN2 ._SENT In_PRP addition_NN1 ,_PUN we_PNP inferred_VVD the_AT0 target_NN1 side_NN1 (_PUL i.e._AV0 ,_PUN petitioner_NN1 or_CJC respondent_NN1 )_PUR of_PRF each_DT0 justice_NN1 's_POS question_NN1 based_VVN on_PRP the_AT0 side_NN1 of_PRF the_AT0 most_AV0 recently_AV0 speaking_VVG lawyer_NN1 ._SENT Additional_AJ0 case_NN1 data_NN0 were_VBD merged_VVN from_PRP the_AT0 Supreme_AJ0 Court_NN1 Database_NN1 (_PUL Spaeth_NP0 et_NP0 al._NP0 Reference_NN1 Spaeth_NP0 ,_PUN Lee_NP0 ,_PUN Ruger_NN1 ,_PUN Whittington_NP0 ,_PUN Segal_NP0 and_CJC Martin2014_NP0 )_PUR ._SENT 
Using_VVG Oyez_NP0 timestamps_NN2 ,_PUN we_PNP segmented_VVD the_AT0 full-argument_AJ0 audio_NN1 into_PRP a_AT0 series_NN0 of_PRF single-speaker_AJ0 utterances_NN2 ._SENT As_CJS an_AT0 additional_AJ0 preprocessing_NN1 step_NN1 ,_PUN we_PNP drop_VVB utterances_NN2 spoken_VVN by_PRP lawyers_NN2 (_PUL each_DT0 of_PRF whom_PNQ usually_AV0 appears_VVZ in_PRP only_AV0 a_AT0 handful_NN1 of_PRF cases_NN2 )_PUR and_CJC Clarence_NP0 Thomas_NP0 (_PUL who_PNQ spoke_VVD only_AV0 twice_AV0 in_PRP our_DPS corpus_NN1 )_PUR ,_PUN focusing_VVG on_PRP the_AT0 behavior_NN1 of_PRF the_AT0 eight_CRD recurrent_AJ0 speakers_NN2 ._SENT We_PNP also_AV0 drop_VVB procedural_AJ0 statements_NN2 ,_PUN along_AVP with_PRP utterances_NN2 shorter_AJC than_CJS 2.5_CRD seconds_NN2 ._SENT After_PRP trimming_VVG and_CJC dropping_VVG cases_NN2 in_PRP which_DTQ the_AT0 ideological_AJ0 direction_NN1 is_VBZ unclear_AJ0 ,_PUN the_AT0 resulting_AJ0 audio_AJ0 corpus_NN1 contains_VVZ 266_CRD arguments_NN2 and_CJC 95_CRD hours_NN2 of_PRF audio_NN1 ,_PUN comprising_VVG nearly_AV0 42_CRD ,_PUN 000_CRD utterances_NN2 and_CJC over_PRP 27_CRD million_CRD frames_NN2 ._SENT 
The_AT0 Quantity_NN1 of_PRF Interest_NN1 :_PUN Judicial_AJ0 Skepticism_NN1 
In_PRP this_DT0 section_NN1 and_CJC the_AT0 next_ORD ,_PUN we_PNP introduce_VVB and_CJC employ_VVB a_AT0 new_AJ0 measure_NN1 of_PRF substantive_AJ0 importance_NN1 to_PRP the_AT0 study_NN1 of_PRF courts_NN2 :_PUN judicial_AJ0 skepticism_NN1 ,_PUN an_AT0 individual_NN1 's_POS vocal_AJ0 expression_NN1 of_PRF their_DPS judgment_NN1 about_PRP the_AT0 argument_NN1 at_PRP hand_NN1 ._SENT Judicial_AJ0 skepticism_NN1 is_VBZ an_AT0 important_AJ0 signal_NN1 of_PRF disagreement_NN1 with_PRP or_CJC incredulity_NN1 about_PRP assertions_NN2 and_CJC legal_AJ0 arguments_NN2 ,_PUN especially_AV0 in_PRP the_AT0 context_NN1 of_PRF oral_AJ0 arguments_NN2 ._SENT 
To_TO0 identify_VVI judicial_AJ0 skepticism_NN1 in_PRP speech_NN1 ,_PUN we_PNP first_ORD randomly_AV0 selected_VVN a_AT0 training_NN1 set_NN1 of_PRF 200_CRD utterances_NN2 per_PRP justice_NN1 to_TO0 hand-classify_VVI as_PRP "_PUQ skeptical_AJ0 "_PUQ or_CJC "_PUQ neutral_AJ0 "_PUQ speech_NN1 ,_PUN allowing_VVG our_DPS assessments_NN2 to_TO0 reflect_VVI not_XX0 only_AV0 the_AT0 vocal_AJ0 tone_NN1 but_CJC also_AV0 the_AT0 textual_AJ0 content_NN1 of_PRF the_AT0 utterance_NN1 ._SENT Thus_AV0 ,_PUN we_PNP define_VVB 16_CRD modes_NN2 of_PRF speech_NN1 —_PUN two_CRD tones_NN2 for_PRP each_DT0 of_PRF the_AT0 eight_CRD speaking_AJ0 justices_NN2 ._SENT During_PRP classification_NN1 ,_PUN we_PNP dropped_VVD the_AT0 handful_NN1 of_PRF utterances_NN2 (_PUL roughly_AV0 5_CRD %_NN0 )_PUR in_PRP which_DTQ crosstalk_NN1 or_CJC other_AJ0 audio_AJ0 anomalies_NN2 occurred_VVN or_CJC in_PRP rare_AJ0 instances_NN2 where_AVQ the_AT0 speaker_NN1 's_POS identity_NN1 was_VBD incorrectly_AV0 recorded_VVN ._SENT The_AT0 model_NN1 is_VBZ then_AV0 estimated_VVN following_PRP Algorithm_NN1 1._CRD 
A_AT0 Case_NN1 Study_NN1 of_PRF Judicial_AJ0 Skepticism_NN1 
To_TO0 illustrate_VVI the_AT0 use_NN1 of_PRF skepticism_NN1 during_PRP the_AT0 flow_NN1 of_PRF oral_AJ0 arguments_NN2 ,_PUN we_PNP conducted_VVD a_AT0 case_NN1 study_NN1 of_PRF Alabama_NP0 Legislative_AJ0 Black_AJ0 Caucus_NN1 v._PRP Alabama_NP0 ,_PUN a_AT0 racial_AJ0 gerrymandering_NN1 case_NN1 heard_VVN by_PRP the_AT0 Supreme_AJ0 Court_NN1 in_PRP 2014_CRD that_CJT considered_VVD the_AT0 legality_NN1 of_PRF Alabama_NP0 's_POS 2012_CRD redistricting_AJ0 efforts_NN2 ._SENT The_AT0 study_NN1 is_VBZ described_VVN in_PRP depth_NN1 in_PRP Online_AJ0 Appendix_NN1 Section_NN1 3_CRD ;_PUN we_PNP briefly_AV0 summarize_VVB it_PNP here_AV0 and_CJC demonstrate_VVB the_AT0 application_NN1 of_PRF MASS_NN1 to_PRP this_DT0 case_NN1 in_PRP Figure_NN1 3._CRD 
As_PRP background_NN1 ,_PUN the_AT0 case_NN1 arose_VVD when_CJS the_AT0 Republican-led_AJ0 legislature_NN1 redrew_VVD electoral_AJ0 districts_NN2 in_PRP the_AT0 face_NN1 of_PRF declining_AJ0 urban_AJ0 population_NN1 ._SENT In_PRP doing_VDG so_AV0 ,_PUN the_AT0 legislature_NN1 sought_VVD to_TO0 pack_VVI Black_AJ0 voters_NN2 into_PRP a_AT0 small_AJ0 number_NN1 of_PRF already_AV0 heavily_AV0 Democratic_AJ0 districts_NN2 ._SENT The_AT0 liberal_AJ0 Alabama_NP0 Legislative_AJ0 Black_AJ0 Caucus_NN1 argued_VVD that_CJT this_DT0 practice_NN1 violated_VVD the_AT0 Voting_NN1 Rights_NN2 Act_NN1 (_PUL VRA_NP0 )_PUR ,_PUN a_AT0 position_NN1 ultimately_AV0 supported_VVN by_PRP the_AT0 Court_NN1 's_POS decision_NN1 ,_PUN whereas_CJS conservative_AJ0 defenders_NN2 of_PRF the_AT0 new_AJ0 map_NN1 argued_VVD that_CJT Section_NN1 5_CRD of_PRF the_AT0 VRA_NN1 in_PRP fact_NN1 forced_VVD the_AT0 legislature_NN1 to_TO0 draw_VVI Black-dominated_AJ0 districts_NN2 ._SENT 
Figure_NN1 3_CRD depicts_VVZ two_CRD exchanges_NN2 in_PRP which_DTQ justices_NN2 spar_NN1 over_PRP this_DT0 legal_AJ0 claim_NN1 about_PRP Section_NN1 5_CRD —_PUN that_CJT a_AT0 state_NN1 must_VM0 hold_VVI or_CJC increase_VVI the_AT0 numerical_AJ0 percentage_NN1 of_PRF Black_AJ0 voters_NN2 in_PRP a_AT0 district_NN1 to_TO0 maintain_VVI minorities_NN2 '_POS "_PUQ ability_NN1 to_TO0 elect_VVI their_DPS preferred_AJ0 candidates_NN2 ._SENT "_PUQ In_PRP the_AT0 first_ORD exchange_NN1 ,_PUN beginning_VVG with_PRP Justice_NP0 Scalia_NP0 's_POS question_NN1 ,_PUN "_PUQ Well_AV0 ,_PUN I_PNP thought_VVD the_AT0 Section_NN1 5_CRD obligation_NN1 ..._PUN ,_PUN "_PUQ Justice_NP0 Scalia_NP0 advocates_VVZ the_AT0 conservative_AJ0 view_NN1 when_CJS questioning_VVG the_AT0 liberal_AJ0 advocate_NN1 ._SENT Median_AJ0 Justice_NP0 Kennedy_NP0 ,_PUN perhaps_AV0 influenced_VVN by_PRP this_DT0 line_NN1 of_PRF questioning_NN1 ,_PUN pursues_VVZ it_PNP further_AJC and_CJC transitions_NN2 into_PRP skepticism_NN1 on_PRP the_AT0 technical_AJ0 point_NN1 of_PRF whether_CJS the_AT0 liberal_AJ0 reading_NN1 of_PRF the_AT0 VRA_NN1 constitutes_VVZ an_AT0 indefensible_AJ0 "_PUQ one-way_AJ0 ratchet_NN1 "_PUQ on_PRP minority_NN1 percentages_NN2 ._SENT Justice_NP0 Breyer_NP0 then_AV0 comes_VVZ to_PRP the_AT0 defense_NN1 of_PRF the_AT0 liberal_AJ0 side_NN1 ,_PUN asking_VVG the_AT0 friendly_AJ0 rhetorical_AJ0 question_NN1 —_PUN ostensibly_AV0 to_PRP the_AT0 advocate_NN1 —_PUN of_PRF whether_CJS Justice_NP0 Kennedy_NP0 's_POS question_NN1 was_VBD addressed_VVN by_PRP precedent_NN1 ._SENT Later_AV0 in_PRP the_AT0 oral_AJ0 argument_NN1 ,_PUN these_DT0 roles_NN2 reverse_VVB ._SENT The_AT0 figure_NN1 depicts_VVZ a_AT0 subsequent_AJ0 exchange_NN1 in_PRP which_DTQ Justice_NP0 Kennedy_NP0 initiates_VVZ a_AT0 line_NN1 of_PRF questioning_NN1 ,_PUN Justice_NP0 Breyer_NP0 attacks_VVZ by_PRP skeptically_AV0 saying_VVG to_PRP the_AT0 conservative_AJ0 advocate_NN1 ,_PUN "_PUQ I_PNP do_VDB n't_XX0 know_VVI what_DTQ the_AT0 defense_NN1 is_VBZ possibly_AV0 going_VVG to_TO0 be_VBI ,_PUN "_PUQ and_CJC Justice_NP0 Scalia_NP0 comes_VVZ to_PRP the_AT0 rescue_NN1 with_PRP a_AT0 "_PUQ softball_NN1 "_PUQ question_NN1 ._SENT 
These_DT0 exchanges_NN2 illustrate_VVB the_AT0 sort_NN1 of_PRF political_AJ0 interactions_NN2 modeled_VVN by_PRP MASS._NP0 Panel_NN1 3.A_NN1 depicts_VVZ how_AVQ skeptical_AJ0 and_CJC neutral_AJ0 speech_NN1 are_VBB deployed_VVN throughout_PRP the_AT0 discussion_NN1 ,_PUN and_CJC Panels_NN2 3.B.1_CRD –_PUN 3_CRD each_DT0 highlight_VVB a_AT0 justice_NN1 's_POS choice_NN1 of_PRF tone_NN1 (_PUL e.g._AV0 ,_PUN the_AT0 decision_NN1 to_TO0 switch_VVI from_PRP neutrality_NN1 to_PRP skepticism_NN1 ,_PUN which_DTQ we_PNP model_VVB using_VVG covariates_NN2 such_DT0 as_PRP conversational_AJ0 history_NN1 or_CJC the_AT0 ideology_NN1 of_PRF the_AT0 side_NN1 currently_AV0 being_VBG questioned_VVN )_PUR ._SENT Panels_NN2 3.C.1_CRD –_PUN 2_CRD examine_VVB two_CRD utterances_NN2 in_PRP depth_NN1 ,_PUN showing_VVG a_AT0 subset_NN1 of_PRF the_AT0 auditory_AJ0 features_NN2 that_DT0 MASS_NN1 relies_VVZ on_AVP to_TO0 infer_VVI speech_NN1 tone_NN1 ._SENT Each_DT0 tone_NN1 is_VBZ modeled_VVN as_PRP a_AT0 sequence_NN1 of_PRF discrete_AJ0 sounds_NN2 ,_PUN like_PRP "_PUQ vowel_NN1 "_PUQ or_CJC "_PUQ silence_NN1 "_PUQ ;_PUN their_DPS usage_NN1 is_VBZ shown_VVN in_PRP Panels_NN2 3.D.1_CRD –_PUN 2_CRD ,_PUN and_CJC their_DPS auditory_AJ0 content_NN1 is_VBZ in_PRP Panel_NN1 3.E_NN1 ._SENT 
Validating_VVG the_AT0 Model_NN1 
We_PNP conduct_VVB extensive_AJ0 validation_NN1 of_PRF our_DPS model-based_AJ0 measure_NN1 of_PRF judicial_AJ0 skepticism_NN1 ,_PUN confirming_VVG that_CJT MASS_NN1 does_VDZ in_PRP fact_NN1 successfully_AV0 estimate_VVB the_AT0 quantity_NN1 of_PRF interest_NN1 ._SENT Due_AJ0 to_PRP space_NN1 constraints_NN2 ,_PUN we_PNP summarize_VVB these_DT0 efforts_NN2 here_AV0 ;_PUN results_NN2 are_VBB described_VVN in_PRP detail_NN1 in_PRP Online_AJ0 Appendix_NN1 Section_NN1 4._CRD 
First_ORD ,_PUN we_PNP demonstrate_VVB that_CJT MASS_NN1 recovers_VVZ a_AT0 measure_NN1 that_CJT has_VHZ high_AJ0 face_NN1 validity_NN1 ._SENT Online_AJ0 Appendix_NN1 4.1_CRD presents_NN2 randomly_AV0 sampled_VVN utterances_NN2 from_PRP the_AT0 highest_AJS and_CJC lowest_AJS deciles_NN2 of_PRF model-predicted_AJ0 skepticism_NN1 ._SENT Those_DT0 characterized_VVN by_PRP the_AT0 model_NN1 as_CJS skeptical_AJ0 include_VVB gentle_AJ0 mockery_NN1 and_CJC doubtful_AJ0 questions_NN2 ,_PUN whereas_CJS model-predicted_AJ0 neutral_AJ0 utterances_NN2 are_VBB factual_AJ0 statements_NN2 and_CJC straightforward_AJ0 legal_AJ0 analysis_NN1 ._SENT 
Second_ORD ,_PUN we_PNP examine_VVB content_AJ0 validity_NN1 in_PRP Online_AJ0 Appendices_NN2 4.2_CRD –_PUN 4.3_CRD ._SENT Our_DPS model_NN1 detects_VVZ skepticism_NN1 based_VVN on_PRP linguistically_AV0 and_CJC physiologically_AV0 meaningful_AJ0 auditory_AJ0 features_NN2 ._SENT In_PRP a_AT0 model_NN1 of_PRF Justice_NP0 Kennedy_NP0 's_POS expressed_AJ0 skepticism_NN1 ,_PUN compared_VVN with_PRP neutral_AJ0 questioning_NN1 ,_PUN we_PNP find_VVB that_CJT his_DPS speech_NN1 is_VBZ louder_AJC ,_PUN contains_VVZ more_DT0 pitch_NN1 variation_NN1 ,_PUN and_CJC is_VBZ characterized_VVN by_PRP higher_AJC vocal_AJ0 tension_NN1 ._SENT We_PNP caution_VVB that_CJT MASS_NN1 does_VDZ not_XX0 take_VVI textual_AJ0 signals_NN2 of_PRF skepticism_NN1 into_PRP account_NN1 ,_PUN an_AT0 important_AJ0 limitation_NN1 on_PRP content_AJ0 validity_NN1 ._SENT (_PUL Joint_AJ0 models_NN2 of_PRF audio_NN1 and_CJC text_NN1 remain_VVB an_AT0 important_AJ0 direction_NN1 for_PRP future_AJ0 work_NN1 ._SENT )_PUR However_AV0 ,_PUN we_PNP demonstrate_VVB that_CJT in_PRP the_AT0 case_NN1 of_PRF judicial_AJ0 speech_NN1 ,_PUN there_EX0 are_VBB extremely_AV0 few_DT0 textual_AJ0 signals_NN2 that_CJT distinguish_VVB skepticism_NN1 from_PRP typical_AJ0 questioning_NN1 ._SENT 
Third_ORD ,_PUN in_PRP Online_AJ0 Appendix_NN1 4.4_CRD ,_PUN we_PNP estimate_VVB a_AT0 lower_AJC bound_NN1 on_PRP the_AT0 out-of-sample_AJ0 performance_NN1 of_PRF MASS_NN1 ,_PUN using_VVG cross-validation_NN1 results_NN2 from_PRP the_AT0 lower_AJC stage_NN1 of_PRF the_AT0 model_NN1 ,_PUN corresponding_VVG to_PRP Equations_NN2 3_CRD –_PUN 4_CRD and_CJC excluding_VVG Equations_NN2 1_CRD –_PUN 2._CRD We_PNP find_VVB that_CJT out-of-sample_AJ0 accuracy_NN1 of_PRF the_AT0 lower-level_AJ0 auditory_AJ0 classifier_NN1 is_VBZ 68_CRD %_NN0 ,_PUN versus_PRP the_AT0 52_CRD %_NN0 that_CJT would_VM0 be_VBI obtained_VVN by_PRP randomly_AV0 permuting_VVG labels_NN2 ;_PUN numerous_AJ0 additional_AJ0 performance_NN1 metrics_NN2 are_VBB reported_VVN in_PRP the_AT0 appendix_NN1 ._SENT 
Fourth_ORD ,_PUN we_PNP compare_VVB the_AT0 performance_NN1 of_PRF MASS_NN1 with_PRP (_PUL 1_CRD )_PUR human_AJ0 coders_NN2 and_CJC (_PUL 2_CRD )_PUR text-based_AJ0 classifiers_NN2 ._SENT For_PRP the_AT0 human_AJ0 comparison_NN1 ,_PUN we_PNP recruited_VVD native_AJ0 English_AJ0 speakers_NN2 on_PRP a_AT0 crowdworking_AJ0 site_NN1 and_CJC evaluated_VVD their_DPS ability_NN1 to_TO0 recover_VVI ground-truth_NN1 labels_NN2 in_PRP legal_AJ0 argumentation_NN1 ._SENT We_PNP found_VVD that_CJT when_CJS combining_VVG responses_NN2 by_PRP majority_NN1 vote_NN1 ,_PUN nonexpert_AJ0 listeners_NN2 were_VBD able_AJ0 to_TO0 detect_VVI 70_CRD %_NN0 of_PRF judicial_AJ0 skepticism_NN1 ,_PUN outperforming_VVG MASS_NN1 by_PRP a_AT0 small_AJ0 margin_NN1 ._SENT The_AT0 performance_NN1 of_PRF individual_AJ0 coders_NN2 was_VBD lower_AJC ,_PUN at_PRP 65_CRD %_NN0 ,_PUN suggesting_VVG that_CJT with_PRP a_AT0 relatively_AV0 small_AJ0 amount_NN1 of_PRF domain-specific_AJ0 data_NN0 ,_PUN our_DPS model_NN1 performs_VVZ approximately_AV0 as_CJS well_AV0 as_CJS humans_NN2 with_PRP a_AT0 lifetime_NN1 of_PRF experience_NN1 in_PRP parsing_VVG non-domain-specific_AJ0 speech_NN1 ._SENT For_PRP the_AT0 textual_AJ0 comparison_NN1 ,_PUN we_PNP applied_VVD an_AT0 elastic_AJ0 net_NN1 to_PRP utterance_NN1 word_NN1 counts_NN2 ._SENT The_AT0 resulting_AJ0 trained_AJ0 text_NN1 models_NN2 were_VBD entirely_AV0 degenerate_AJ0 ,_PUN predicting_VVG the_AT0 more_AV0 common_AJ0 label_NN1 in_PRP virtually_AV0 every_AT0 case_NN1 ._SENT 
Finally_AV0 ,_PUN we_PNP probe_VVB the_AT0 predictive_AJ0 validity_NN1 of_PRF our_DPS measure_NN1 with_PRP a_AT0 comparison_NN1 to_PRP Black_NP0 et_NP0 al._NP0 (_PUL Reference_VVB Black_NP0 ,_PUN Treul_NP0 ,_PUN Johnson_NP0 and_CJC Goldman2011_NP0 )_PUR ,_PUN described_VVN in_PRP the_AT0 next_ORD section_NN1 ._SENT 
Comparison_NN1 with_PRP an_AT0 Existing_AJ0 Measure_NN1 
We_PNP conduct_VVB yet_AV0 another_DT0 validity_NN1 test_NN1 by_PRP contrasting_VVG our_DPS model_NN1 with_PRP the_AT0 approach_NN1 of_PRF Black_NP0 et_NP0 al._NP0 (_PUL Reference_VVB Black_NP0 ,_PUN Treul_NP0 ,_PUN Johnson_NP0 and_CJC Goldman2011_NP0 )_PUR ,_PUN who_PNQ use_VVB a_AT0 measure_NN1 based_VVN on_PRP directed_AJ0 pleasant_AJ0 and_CJC unpleasant_AJ0 words_NN2 —_PUN operationalized_VVN with_PRP the_AT0 Dictionary_NN1 of_PRF Affect_NN1 in_PRP Language_NN1 (_PUL DAL_NN1 ;_PUN Whissell_NP0 Reference_NN1 Whissell2009_CRD )_PUR —_PUN to_TO0 predict_VVI justice_NN1 voting_NN1 ._SENT We_PNP replicate_VVB and_CJC contrast_VVB results_NN2 with_PRP those_DT0 from_PRP a_AT0 comparable_AJ0 measure_NN1 of_PRF directed_AJ0 skepticism_NN1 ._SENT Specifics_NN2 are_VBB discussed_VVN in_PRP Online_AJ0 Appendix_NN1 4.5_CRD ._SENT We_PNP find_VVB that_CJT a_AT0 one-standard-deviation_AJ0 increase_NN1 in_PRP directed_AJ0 unpleasant_AJ0 (_PUL pleasant_AJ0 )_PUR words_NN2 is_VBZ associated_VVN with_PRP a_AT0 2.8-percentage-point_AJ0 decrease_NN1 (_PUL no_AT0 difference_NN1 )_PUR in_PRP the_AT0 probability_NN1 that_CJT a_AT0 justice_NN1 votes_NN2 against_PRP a_AT0 side_NN1 ._SENT In_PRP comparison_NN1 ,_PUN a_AT0 one-standard-deviation_AJ0 increase_NN1 in_PRP directed_AJ0 skepticism_NN1 ,_PUN as_CJS estimated_VVN from_PRP auditory_AJ0 characteristics_NN2 ,_PUN is_VBZ associated_VVN with_PRP a_AT0 7.9-percentage-point_AJ0 decrease_NN1 in_PRP vote_NN1 probability_NN1 —_PUN nearly_AV0 three_CRD times_NN2 as_CJS large_AJ0 ._SENT Moreover_AV0 ,_PUN Figure_NN1 4_CRD shows_VVZ that_CJT unlike_PRP text-based_AJ0 results_NN2 ,_PUN these_DT0 patterns_NN2 are_VBB robust_AJ0 to_PRP the_AT0 inclusion_NN1 of_PRF both_DT0 justice_NN1 and_CJC case_NN1 fixed_AJ0 effects_NN2 ._SENT 
Why_AVQ is_VBZ speech_NN1 tone_NN1 so_AV0 much_AV0 more_AV0 predictive_AJ0 of_PRF voting_VVG patterns_NN2 than_CJS the_AT0 use_NN1 of_PRF affective_AJ0 words_NN2 ?_SENT One_CRD reason_NN1 may_VM0 be_VBI that_DT0 DAL_NN1 uses_VVZ a_AT0 cross-domain_AJ0 measure_NN1 of_PRF word_NN1 pleasantness_NN1 ,_PUN ignoring_VVG the_AT0 fact_NN1 that_CJT words_NN2 often_AV0 take_VVB on_PRP different_AJ0 meanings_NN2 in_PRP legal_AJ0 contexts_NN2 ._SENT For_PRP example_NN1 ,_PUN the_AT0 the_AT0 10_CRD most_AV0 common_AJ0 "_PUQ unpleasant_AJ0 "_PUQ words_NN2 in_PRP our_DPS corpus_NN1 include_VVB "_PUQ argument_NN1 "_PUQ and_CJC "_PUQ trial_NN1 ,_PUN "_PUQ whereas_CJS the_AT0 five_CRD most_AV0 common_AJ0 "_PUQ pleasant_AJ0 "_PUQ words_NN2 include_VVB "_PUQ read_VVN "_PUQ and_CJC "_PUQ justice_NN1 ._SENT "_PUQ However_AV0 ,_PUN as_CJS we_PNP show_VVB in_PRP other_AJ0 text-audio_NN1 comparisons_NN2 ,_PUN a_AT0 more_AV0 likely_AJ0 explanation_NN1 is_VBZ that_CJT word_NN1 choice_NN1 is_VBZ a_AT0 noisy_AJ0 ,_PUN high-dimensional_AJ0 ,_PUN and_CJC difficult-to-measure_AJ0 signal_NN1 of_PRF expressed_AJ0 emotion_NN1 ,_PUN whereas_CJS auditory_AJ0 tone_NN1 is_VBZ relatively_AV0 structured_AJ0 and_CJC consistent_AJ0 ._SENT 
However_AV0 ,_PUN we_PNP note_VVB that_CJT MASS_AJ0 exploits_NN2 only_AV0 the_AT0 auditory_AJ0 channel_NN1 of_PRF speech_NN1 ._SENT While_CJS we_PNP show_VVB in_PRP Online_AJ0 Appendix_NN1 4.4_CRD that_CJT this_DT0 provides_VVZ a_AT0 clearer_AJC signal_NN1 of_PRF skepticism_NN1 than_CJS text_NN1 in_PRP general_AJ0 ,_PUN there_EX0 are_VBB nonetheless_AV0 cases_NN2 when_CJS expressions_NN2 of_PRF disbelief_NN1 are_VBB spoken_VVN flatly_AV0 ._SENT In_PRP one_CRD example_NN1 ,_PUN Justice_NP0 Ginsburg_NP0 states_VVZ matter-of-factly_AV0 ,_PUN "_PUQ Well_AV0 ,_PUN that_DT0 would_VM0 be_VBI fine_AJ0 if_CJS the_AT0 statute_NN1 said_VVD what_DTQ you_PNP claim_VVB it_PNP said_VVD ._SENT "_PUQ The_AT0 utterance_NN1 is_VBZ clearly_AV0 skeptical_AJ0 ,_PUN yet_AV0 our_DPS model_NN1 predicts_VVZ that_CJT it_PNP is_VBZ 89_CRD %_NN0 likely_AJ0 to_TO0 be_VBI neutral_AJ0 speech_NN1 due_AJ0 to_PRP its_DPS deadpan_NN1 delivery_NN1 ._SENT This_DT0 counterexample_NN1 highlights_VVZ limitations_NN2 in_PRP the_AT0 use_NN1 of_PRF any_DT0 communication_NN1 channel_NN1 in_PRP isolation_NN1 ,_PUN suggesting_VVG that_CJT joint_AJ0 models_NN2 of_PRF text_NN1 and_CJC tone_NN1 are_VBB a_AT0 necessary_AJ0 direction_NN1 for_PRP future_AJ0 work_NN1 ._SENT 
Next_ORD ,_PUN we_PNP demonstrate_VVB the_AT0 model_NN1 with_PRP an_AT0 application_NN1 to_PRP speech_NN1 in_PRP Supreme_AJ0 Court_NN1 oral_AJ0 arguments_NN2 ,_PUN then_AV0 conclude_VVB ._SENT 
Testing_NN1 Theories_NN2 of_PRF Supreme_AJ0 Court_NN1 Deliberation_NN1 
While_CJS some_DT0 scholars_NN2 believe_VVB that_CJT oral_AJ0 arguments_NN2 are_VBB inconsequential_AJ0 in_PRP Supreme_AJ0 Court_NN1 decision-making_NN1 (_PUL Rohde_NP0 and_CJC Spaeth_NP0 Reference_VVB Rohde_NP0 and_CJC Spaeth1976_NP0 ;_PUN Segal_NP0 and_CJC Spaeth_NP0 Reference_VVB Segal_NP0 and_CJC Spaeth1993_NP0 )_PUR ,_PUN others_NN2 argue_VVB that_CJT they_PNP play_VVB a_AT0 crucial_AJ0 role_NN1 in_PRP shaping_VVG the_AT0 Court_NN1 's_POS ultimate_AJ0 position_NN1 on_PRP a_AT0 case_NN1 (_PUL Black_NP0 ,_PUN Sorenson_NP0 ,_PUN and_CJC Johnson_NP0 Reference_VVB Black_NP0 ,_PUN Sorenson_NP0 and_CJC Johnson2013_NP0 ;_PUN Epstein_NP0 ,_PUN Landes_NP0 ,_PUN and_CJC Posner_NP0 Reference_VVB Epstein_NP0 ,_PUN Landes_NP0 and_CJC Posner2010_NP0 ;_PUN Johnson_NP0 Reference_NN1 Johnson2001_NP0 ;_PUN Johnson_NP0 ,_PUN Wahlbeck_NP0 ,_PUN and_CJC Spriggs_NP0 Reference_VVB Johnson_NP0 ,_PUN Wahlbeck_NP0 and_CJC Spriggs2006_NP0 ;_PUN McGuire_NP0 Reference_NN1 McGuire1995_NP0 ;_PUN Shapiro_NP0 Reference_NN1 Shapiro1984_NP0 ;_PUN Wasby_NP0 ,_PUN D'Amato_NP0 ,_PUN and_CJC Metrailer_NP0 Reference_VVB Wasby_NP0 ,_PUN D'Amato_NP0 and_CJC Metrailer1976_NP0 )_PUR ._SENT The_AT0 justices_NN2 themselves_PNX ,_PUN however_AV0 ,_PUN are_VBB virtually_AV0 unanimous_AJ0 on_PRP this_DT0 point_NN1 ._SENT Justice_NP0 Powell_NP0 stated_VVD ,_PUN "_PUQ The_AT0 fact_NN1 is_VBZ ,_PUN as_CJS every_AT0 judge_NN1 knows_VVZ ..._PUN the_AT0 oral_AJ0 argument_NN1 ..._PUN does_VDZ contribute_VVI significantly_AV0 to_PRP the_AT0 development_NN1 of_PRF precedents_NN2 "_PUQ (_PUL Stern_NP0 ,_PUN Gressman_NP0 ,_PUN and_CJC Shapiro_NP0 Reference_NN1 Stern_NN1 ,_PUN Gressman_NP0 and_CJC Shapiro1993_NP0 ,_PUN 571_CRD )_PUR ._SENT Johnson_NP0 and_CJC Goldman_NP0 (_PUL Reference_VVB Johnson_NP0 ,_PUN Goldman_NP0 ,_PUN Johnson_NP0 and_CJC Goldman2009_NP0 )_PUR document_NN1 numerous_AJ0 other_AJ0 direct_AJ0 quotes_NN2 about_PRP the_AT0 importance_NN1 of_PRF oral_AJ0 arguments_NN2 ,_PUN including_PRP Justices_NP0 Harlan_NP0 ,_PUN Hughes_NP0 ,_PUN Jackson_NP0 ,_PUN Brennan_NP0 ,_PUN White_NP0 ,_PUN Rehnquist_NP0 ,_PUN and_CJC others_NN2 ._SENT 
But_CJC there_EX0 is_VBZ little_DT0 empirical_AJ0 evidence_NN1 about_PRP how_AVQ arguments_NN2 matter_VVB ._SENT Courts_NN2 scholars_NN2 have_VHB advanced_VVN various_AJ0 accounts_NN2 of_PRF the_AT0 deliberative_AJ0 process_NN1 ,_PUN which_DTQ can_VM0 be_VBI roughly_AV0 grouped_VVN into_PRP two_CRD broad_AJ0 theories_NN2 ._SENT For_PRP concreteness_NN1 ,_PUN we_PNP discuss_VVB these_DT0 theories_NN2 and_CJC their_DPS associated_AJ0 mechanisms_NN2 in_PRP the_AT0 context_NN1 of_PRF Rapanos_NP0 v._PRP United_NP0 States_NP0 ,_PUN a_AT0 narrowly_AV0 decided_VVN environmental_AJ0 case_NN1 about_PRP the_AT0 Army_NN1 Corps_NN0 of_PRF Engineers_NN2 '_POS right_NN1 to_TO0 regulate_VVI pollution_NN1 in_PRP wetlands_NN2 ,_PUN drawing_VVG extensively_AV0 on_PRP legal_AJ0 analysis_NN1 by_PRP Savage_NP0 (_PUL Reference_VVB Savage_NP0 ,_PUN Johnson_NP0 and_CJC Goldman2009_NP0 )_PUR ._SENT 
