Tg) and storage modulus. These properties have been compared with previously reported values using pristine graphene (Gr) as a filler. The increase in thermal mechanical properties on addition of silanized graphene (SiGr) showed a large shift in the Tg and more increase in storage modulus by chemically binding SiGr sheets on the aramid chains. Aminophenyl-trialkoxysilane was found to give better results due to the presence of phenyl groups which were more rigid than propyl groups present in aminopropyl-trialkoxysilane. The effect of chemical bonding and the possible π–π secondary bond interactions between the matrix and graphene platelets on the properties of the resulting hybrids are discussed.The synthesis and characterization of aramid composites reinforced with graphene platelets are reported. Hydroxy-functionalised graphene platelets were modified with two sol–gel binders  and then chemically linked with aramid chains. The effect of the two sol–gel binders on the physiochemical and mechanical properties was evaluated. Chemical changes during the sol–gel reaction and subsequent amidation process in the nano-composite preparation were evaluated by the XPS and FTIR analyses. Thin films of these composites with different proportions of graphene were prepared. Morphology of the hybrids prepared was studied by the SEM technique. Properties of the composite films were studied by dynamical mechanical thermal (DMT) analysis to measure their glass transition temperature ( The synthesis and characterization of aramid composites reinforced with graphene platelets are reported. Polymer composites can provide high mechanical strength and thermal stability with diverse properties which are substantially different or even better as required for the advanced applications. This is possible, in particular, when the filler is present at the nanoscale5–7 and is distributed homogenously in the matrix. Due to the unique and tunable properties of network structures, the use of inorganic materials in combination with high performance polymers has attracted a lot of interest in the recent past for numerous applications in the aerospace and electronics industries.8–10 A good number of the publications have been focussed on nano-inorganic particles produced by in situ sol–gel processes11,12 in the matrix or as surface coating materials. The layered silicate structures13–15 using different types of clays also became very popular to enhance the toughness of the polymeric materials. Carbon-based nano-fillers in the recent times, however, are being preferred in comparison to clay or other inorganic metal oxides to improve the mechanical strength of polymers. Polymer composites using carbon black16,17 carbon nanotubes (CNTs)18–24 carbon fibre25,26 and graphene (Gr)27–29 have shown a large improvement in thermal, mechanical, conductive and flame-retardant properties.The last few decades have witnessed a rapid development in the field of polymer composites18–24,30,31 Chemical functionalization of CNTs can increase their interaction with the polymeric matrix facilitating their processability.21,32 The presence of a suitable solvent can greatly enhance their de-aggregation required for a uniform dispersion33 in the matrix. The present authors have shown that strong interfacial interactions of the matrix with the surface-modified CNTs reduced the stress-transfer problem in the composite material and resulted in higher modulus of 4.26 GPa and a glass transition temperature of 338.5 °C, whereas the CTE was reduced to 101.8 ppm on addition of only 2.5 wt% CNTs in aramid matrix24,35. Because of their high aspect ratio, the alignment of the CNTs in a matrix can create a large polymer–filler interface thus producing large reinforcement. But one of the major disadvantages with CNTs is their entanglement tendencies due to which the increased viscosity with higher loadings may lead to a significantly poorer dispersion.34 The alignment of CNTs, even at a laboratory scale35 seems difficult to achieve at present. Another disadvantage with CNTs is their higher cost of production which may discourage their uses for production of such composites on the industrial scale.31,36 Graphene on the other hand is cheaper in comparison to CNTs with exceptionally good mechanical, thermal and electrical properties. Owing to their high surface area, large aspect ratio and high tensile strength and thermal conductivity these can be very suitable to improve the properties of polymeric matrices.27,37,38 It is also much easier to process these platelets at higher loadings in the matrix, as the viscosity of the polymer–graphene mixture is significantly lower than the CNT composites. These platelets due to their micron-size wrinkled layered structure have been found to be a highly efficient load bearer filler in comparison to the CNTs for the polymer matrices.39 The improvement in the ultimate physicochemical characteristics of such composites, however, depends on the distribution of Gr layers in the polymer matrix as well as on the interfacial bonding between the Gr and the matrix.40 Modification of the Gr surface is usually carried out by oxidation followed by its chemical functionalization.41 The oxidized graphene (GrO) bearing various polar i.e., hydroxyl and carboxyl functional groups can alter the van der Waals interactions significantly. These methods make Gr more compatible with organic polymers.42 The covalent or the noncovalent functionalization facilitate dispersion and stabilize Gr in the matrix by preventing its agglomeration.43–45 For this reason, GrO is now being used extensively in industry as a nanofiller in the polymer composites.46,47Among the carbon materials, CNTs have proven to be very effective as conductive fillers and a wide variety of CNT–polymer composites using different techniques have been reported.et al.48 modified GrO surfaces using tetraethoxysilane and the silanized graphene (SiGr) was found to improve the protective properties of composite coatings. High performance benzoxazine–bismaleimide resin based SiGr composites were prepared by Jiang et al.49 who showed that SiGr has good dispersity in the matrix. Thermal decomposition temperature and char yield of the composites were reported to improve markedly. Organically modified silanes represented by M–Si–(OR)3 can be used through the sol–gel processing to link the oxidized graphene with polymer chain. The trialkoxy (OR) can be easily hydrolysed and condensed using sol–gel process and linked with hydroxy functionalized Gr. Depending on the nature of the functional group M, the silanized graphene then can be linked to a polymer chain. The silica network produced during the silanization process can greatly increase the interlayer distance and creates a disorder in the Gr lattice thus ensuring a uniform distribution in the matrix. The GrO was modified by Liu et al.50 using γ-glycidoxypropyltrimethoxysilane to prepare the polyurethane composites. The SiGr provided a good compatibility with the matrix thus increasing the thermal decomposition temperature of polyurethane matrix.The GrO surface can be further modified by the sol–gel process using different types of silanes. The silica network structure can help to increase the interlayer distance thus reducing the agglomeration tendencies of the Gr platelets. Jing Li 51 due to their high specific strength and thermal stability with a lower density are considered as very important commercial fibers. In the present work thermal mechanical properties of such easily processable aromatic polyamide chains were improved by reinforcement with SiGr using two different types of organically modified silanes i.e., aminophenyltrimethylsilane APhTMS and aminopropyltriethoxysilane APrTES. To improve the solubility (processability) of poly chains, generally known as Kevlar© in industry, they were modified by introducing an optimum number of meta-linkages in the chains. The sol–gel process was used to bond the aminosilanes on the surface of hydroxy functionalised graphene, which then acted as a link between the graphene platelets and the polymer matrix chain. Viscoelastic properties of these chemically bonded hybrids with different proportions of SiGr have been determined and compared with our earlier work with those hybrids, which were reinforced with the pristine Gr.43Aromatic polyamidespara- and meta-phenylenediamines (99%) and aminopropyltriethoxysilane (98%) (APrTES), HCl and the solvent dimethyl acetamide (DMAc) with water contents less than 0.004%, all of analytical reagents (AR) grades were procured from Sigma-Aldrich. Graphene oxide powder (99.4%), (plate diameter Ø: 1.5–5.5 μm and thickness: 0.43–1.23 nm) was obtained from Nano US Research Nano materials. 3-Aminophenyltrimethoxysilane (90%) (APhTMS) was obtained from Gelest.The monomers terephthaloyl chloride (99%), meta- and para-phenylenediamines taken respectively in the molar ratio 35 : 65 was placed in a 300 ml conical flask in a glove box under complete anhydrous conditions. The total moles of amines taken were 0.025 and, in this mixture, 110 g of DMAc as solvent was added. The mixture was stirred with magnetic stirrer for 30 min until dissolved completely. Under complete anhydrous condition terephthaloyl chloride (0.025 mol) was then added to react with phenylenediamines. Stirring of the reaction mixture continued for the 24 h to complete the polymerization reaction. The –COCl terminated aramid matrix chains were prepared by further addition of 2.8 millimole of terephthaloyl chloride in the reaction mixture. Further stirring continued for 3 h to complete the endcapping of the chains with –COCl groups.A mixture of i.e., APhTMS and APrTES were used in this work  and 2.8 millimole of each silane was separately added to the graphene solution and the mixture was stirred for 30 min for complete mixing. To this solution, water was added in the form of DMAc solution (taken in 5 : 95 ratio respectively by weight) to carry out the sol–gel reaction. The ratio of silane to water was kept respectively at 1 : 1.5. Few drops of HCl (0.05 M) were added to catalyze the sol–gel reaction. The mixture was continuously stirred at 70 °C for 4 h on a magnetic hotplate. The stirring further continued at room temperature for 24 h. The silanized graphene solution was used as stock solution to prepare (The oxidized graphene (0.30 g) was taken in 20 g DMAc in a 150 ml flask and stirred for 24 h then followed by 2 h sonification. The sol–gel process was then carried out to functionalize the surface of graphene with aminosilanes. Two different types of silanes  prepare  the silaDifferent amounts of two silanized graphene solutions were added separately to a known amount of ClOC-endcapped polymer solution with stirring continued for another 24 h to get the complete mixing . Thin hy−1 using the JASCO FTIR Spectrometer-6300 to monitor the silanization procedure. The composition of different elements i.e., carbon, oxygen, nitrogen and silicon present on the film surface were determined by XPS to identify the functional groups present during the preparation of the hybrid materials. Thermo ESCALAB 250 Xi with a monochromatic radiation source was used for this purpose. FESEM was using to study the morphology of the composite films. The films fractured using the microtome at very low temperature and coated with gold were examined at 15 kV using SEM User, Interface with Model: SUPRA 50 VP(JDISS). The visco-elastic properties of the hybrid films were measured on Dynamic Mechanical Analyzer Q-800 (TA instruments). The storage modulus of the hybrid films at different loadings of SiGr were measured as function of temperature. The glass transition temperature was measured from the peak of tan δ plot vs. temperature. The measurements were taken under tension mode in the temperature range 50–500 °C, at the heating rate of 5 min °C−1 using frequency 5 Hz under nitrogen gas at the floating pressure of 60 Pa. Thermogravimetry was performed on approximately 10 mg of the sample from ambient to 800 °C at a heating rate of 10 °C min−1 in a dynamic nitrogen atmosphere (50 ml min−1), using TGA-50 TA automatic analyzer.The absorption spectra were obtained for the pristine and oxidized Gr, SiGr, ArPhSiGr and ArPrSiGr hybrid films in the range of 400–4000 cm−1 for  scale" fill="currentColor" stroke="none"><path d="M0 440 l0 -40 320 0 320 0 0 40 0 40 -320 0 -320 0 0 -40z M0 280 l0 -40 320 0 320 0 0 40 0 40 -320 0 -320 0 0 -40z"/></g></svg>C) bonds in spectrum (A) shows the presence of the hexagonal structure of pristine Gr whereas the peaks significant in spectrum (B) for GrO correspond to the functionalities created by the oxidation of Gr. A broad absorption peak between the 2600 cm−1 and 3500 cm−1 is mainly related to the hydroxyl groups on the GrO that were used in the present work for the sol–gel reaction. Two small peaks at 2921 cm−1 and 2846 cm−1 can be attributed to the (C–H) asymmetric and symmetric stretching vibrations derived from long alkyl chain of Gr. The adsorption band at 1730 cm−1 and 1625.6 cm−1 correspond to the CO stretching (from carbonyl and carboxyl groups) and aromatic CC vibration that results from skeletal vibration from unoxidized graphitic domains respectively. Besides, two weak peaks present at 1167 cm−1 and at 1051 cm−1 are from the C–O stretching deformation mode of the carboxylic acid group and epoxide functional group respectively. The peak at 1368 cm−1 can be attributed to single bonded carboxyl group (C–O)–O. The high intensity of the peaks due to oxygen functional groups is useful to develop further links of GrO platelets surface with silica by the sol–gel processing and then through amine groups with the matrix chain .52 The new bands at 1116 cm−1 and 1034 cm−1 are, therefore, assigned to the Si–O–Si network. The study conducted by Musto et al.53 has shown that Si–O–Si absorption at higher wave numbers represent the cyclic whereas at the lower wave number the linear silica structure. The absorption at 914 cm−1 is due to Si–OH bonds. The presence of APhTMS in the spectrum (C) is also confirmed by presence of the Si–C bond appearing at 720 cm−1.54 The decrease of broad peak of carboxylic –OH and C–OH around 3000–3500 cm−1 means that the amount of silane used was sufficient to cover the majority of the surface of graphene.Spectrum (C) in −1, 1543 cm−1 and 1320 cm−1 are attributed to amide functional groups which are from aramid and from the amidation reaction between the amine group of APhTMS and –COCl groups of aramid whereas the peak at 3315 cm−1 is due to –NH stretching. The new composite bands at 1119 cm−1 and higher wavenumber and at 864 cm−1 are assigned to the and Si–O–Si and Si–OH respectively. A small peak at 3449 cm−1 seems due to OH groups from the Si–OH produced during sol–gel reaction. The FTIR results shown in the spectra (C) and (D) approves the silanization process. The results obtained for the silanization process and the hybrids using APrTES were of similar nature, and these are not presented here to avoid the repetition chemical binding between the graphene and amide chain has further been investigated by the XPS analysis.The spectrum (D) in the 2 carbon bonds, C–O bonds, CO bonds respectively. In the spectrum (A) as expected N 1s peak is absent.XPS analysis was employed to elucidate the surface composition of Gr, GrO, SiGr, aramid and the hybrid films to provide more information on the interactions between silane coupling agent and the bonding between SiGr and aramid matrix chains. In the XPS spectrum (A) obtained for GrO, the peaks at 284.68 eV, 286.79 eV and 288.43 eV are attributed to the spi.e., from 29.43% to 7.96%. The peak from unreacted –NH2 groups of APhTMS is still present at 401.8 eV which indicates the silanization reaction occurred through alkoxy groups of APhTMS and the NH2 groups are retained.The spectrum (B) shows the presence of both Si and N elements in the SiGr and the concomitant reduction of carbon atoms with respect to GrO that confirm the silane functionalization by APhTMS through the reaction of the alkoxy group of the silane with –OH groups of GrO. Additional peaks at 283.97 eV and 285.37 can be assigned to C bonded to Si owing to the reaction with silane coupling agent. The atomic wt% of carbonyl carbon structure in the SiGr in comparison to GrO decreased considerably C bond. The (C–N) bond formed in the aramid chain or (C–N)–Ar appears at 287.90 eV. The unreacted COCl on aramid chains are confirmed in the carbon spectrum at 289.08 eV. Oxygen peaks from OC–C appears at 531.2 whereas OC–N can be seen at 532.06 eV. The unreacted terminal acid chloride peak OC–Cl is seen at 533.38 eV and N–C and N–H peaks can be at seen at 399.51 eV and at 400.05 eV. In spectrum (D) for ArPhSiGr-8 hybrid film the disappearance of COCl at 289.08 eV and OC–Cl at 533.38 eV and that of –NH2 peak at 401.79 eV confirms the reaction taking place between the acid chloride terminated aramid resin and silanized Gr. The atomic wt% of OC–N in the composite increased as well due to the amide bond formation. The peak of the binding energy (BE) for a specific element depends on the structure and local chemical environment of that element. The OC–C peaks from GrO and silanized Gr are at 532.5 eV and that from neat aramid and its hybrids at 531.2 eV and 531.23 eV respectively. The BE values in the hybrid film are closer to aramid since the matrix contributed more than 92 wt% of the hybrid system.The XPS spectrum (C) for the aramid chains end-capped with –COCl groups shows peaks at 284.67 eV for C–C as well as 285.11 eV for the CScanning electron microscopy imaging was used to evaluate the surface morphology and graphene distribution in the matrix of the composite films produced from two different aminosilanes. Comparing the morphology of the films obtained using APhTMS  with thoE′) and tangent psi (tan δ) as a function of the temperature. The temperature dependence of tan δ can reveal different relaxation processes in the material denoted by α, β and γ transitions. δ curve for ArPhSiGr (δ curve for neat aramid and this peak corresponds to the α-relaxation temperature (Tg). With increase in the loading of silanized graphene the maxima of the curve shifts to higher temperature in both types of hybrids indicating that the reduced mobility of the polymer chains. The surface of graphene is covered with porous silica network and the polymer chains are not only linked with it chemically, but the chains were imbibed within this network as seen in the SEM images. The strong adhesion of the polymer chains with SiGr is also confirmed from the dampening of the tan d curves, which increases as the elastic nature of the material increases as SiGr is increased in the matrix. The Tg for the pure polymer increased from 330 °C to 379 °C for the ArPrSiGr system and to 389 °C for the ArPhSiGr system with 8% SiGr. The increment in Tg is attributed to the suppressed mobility of aramid back bone due to the increased interfacial interaction and the chemical bonding of the polymer chain and on the surface of Gr. This can be attributed to the presence of silica network produced by the sol–gel process has separated the sheets apart thus increasing its free surface to interact with polymer matrix. Chemical bonding between the amine groups of the silanized graphene and the acid chloride end-groups from aramid matrix has further improved their adhesion with the matrix resulting an increase in the Tg of the hybrid material.Thermal mechanical properties of aramid-silanized graphene hybrid films were studied by measuring the stress response during the exposure of the film to a sinusoidal strain within the temperature range of 25 to 500 °C. The results were obtained in terms of the storage modulus , present on both sides of the Gr sheets can have multiple π–π interactions with phenyl rings of aramid matrix we used. Even using pristine Gr, therefore, we found an increase in the Tg on loading with pristine Gr. A comparison in Fig. S2Tg is, therefore, observed when SiGr used as fillers (Liao  fillers .E′) for the two types of composites i.e., ArPrSiGr and ArPhSiGr with different amounts of SiGr is shown in E′ values above 480 °C is due to a softening of the polymer.The effect of temperature on the storage modulus  chain known as Kevlar© has a thermal decomposition temperature around 528 °C.56 To make this type of fiber processable we introduced nearly 20% meta-linkages and a thermal decomposition temperature of the matrix was slightly reduced (A) to 512 °C  and (C) in By a careful choice of choosing an organically modified silane, the silica network structure can be produced by the sol–gel process on graphene surface which help to increase the interlayer distance thus disaggregating the graphene platelets and ensuring their uniform distribution in the polymer matrix. With an appropriate choice of functional groups present on silane it can further be utilized to develop chemical bonding between the filler and the matrix. Greater adhesion between the silanized graphene using 3-aminophenyltrimethoxysilane and the aramid chain resulted in a much higher increase in the glass transition temperature and storage modulus, and it was possible to reinforce the polymer with higher contents of filler.There are no conflicts to declare.RA-012-D2RA04797G-s001


The presented work is devoted to the study of the formation of the thinnest diamond film (diamane). We investigate the initial stages of diamond nucleation in imperfect bilayer graphene exposed by the deposition of H atoms . We show that defects serve as nucleation centers, their hydrogenation is energy favorable and depends on the defect type. Hydrogenation of vacancies facilitates the binding of graphene layers, but the impact wanes already at the second coordination sphere. Defects influence of 5|7 is lower but promotes diamondization. The grain boundary role is similar but can lead to the final formation of a diamond film consisting of chemically connected grains with different surfaces. Interestingly, even hexagonal and cubic two-dimensional diamonds can coexist together in the same film, which suggests the possibility of obtaining a new two-dimensional polycrystal unexplored before. Diamond is probably the best-known crystalline compound of carbon. Diamond nanostructures of different dimensions have also attracted much attention along with the two-dimensional diamond or diamane , which i2, we can expect the appearance of a significant nucleation barrier which can be overcome only by high pressure and temperature, for instance, in the case of single-layer graphene hydrogenation is problematic [The most promising way to obtain a two-dimensional diamond seems to be the use of graphene as a precursor, by deposition of reference atoms  on its surface. In this case the thermodynamic stability of the material is reversed, the previously unstable diamond film becomes energy favorable, and graphene layers tend to bond to each other . Despiteblematic ,14.However, the real structure of graphene contains structural defects that can be used as nucleation centers which may allow the synthesis of diamane under less severe conditions. Here we investigate such an effect in detail. For this purpose, we study one of the most common structural defects in graphene and reveal their impact on diamond nucleation.−5 a.u. while maximum force components were found less than 10−4 Hartree/Bohr.An isolated bigraphene cluster was chosen as the model system. The edge atoms of bigraphene were passivated by hydrogen atoms. The flake diameter was about 30 Å which made it possible to neglect the influence of edge effects on the process of nucleation in the flake center. For geometry relaxation and calculation of the structure energy, the DFTB method was used . In this2 molecule calculated in the framework of DFTB differs from the reference DFT data. Thus, we used in our calculations the reference value of difference (0.23 eV/atom) between the H-H and C-H binding energy for H2 molecule and infinite graphane [The bonding energy of the Hgraphane , respectn chemisorbed H atoms the average formation energy εb (n) is gE is the energy of either monolayer or bilayer graphene substrate, εH is the energy of a single H atom, and nHEg@ is the total energy of the hydrogenated structure.The growth of the diamond phase in multilayer graphene has a nucleating character . This me2 molecule .b (n) intersection with The presence of several nearby vacancies can facilitate diamond formation. In contrast to the vacancy located only in one layer, we considered the agglomeration of cross layer vacancies in bilayer AB stacked graphene which can be obtained by its irradiation with low-energy ion beams of high density. In this structure, the defects affect each other. It leads to the formation of a reactive area between them which easily binds hydrogen atoms and forms interlayer bonds. We considered an agglomerate of vacancies separated from each other by about 5 Å, as seen in In the case of the Stone–Wales (SW) defect commonly observed in graphene , we founNote that the Stone–Wales defect is a constituent part of the grain boundary (GB) in polycrystalline graphene connecting the graphene domains with different orientations . Since sThus, hydrogenation appears to be a prospective way to obtain a specific two-dimensional diamond structure that combines different surfaces. The grain boundary energy of the studied junction is ~1.3 eV/Å, which is only slightly higher than other considered two-dimensional carbon interfaces, namely graphene (<0.4 eV/Å ) and graIn summary, we found that type and concentration of structural defects can sufficiently impact the initial stages of diamond nucleation. At the same time, it does not influence further diamond formation. Defects impact on C-H bonding strength is disappearing at the second coordination sphere already. We show that vacancies agglomeration (that can be produced by low energy ion irradiation) can sufficiently expand the reactive region which vanishes the nucleation barrier for the first stages of nucleation. Stone–Wales defects impact is lower but promotes the hydrogenation and bonding of the graphene layers. We show that 1D defect (dislocation) not only facilitates the diamondization but also may lead to the appearance of 2D diamond consisting of chemically connected grains of different crystallographic orientations. Therefore, polycrystalline graphene usually observed in the experiment can produce specific 2D diamond polycrystals containing different surfaces. Even hexagonal and cubic 2D diamonds can coexist together in the same film with grain boundary energy comparable with the same values for other two-dimensional carbon structures.Our study can be further expanded by more detailed investigation of the thermodynamic stability of formed diamond clusters and the explicit calculations of nucleation barrier dependence on a pressure. Other possible structural defects  as well as multilayer graphene (more than two layers) should be also considered in further work.We believe that the present study will help in further research on the diamondization of multilayer graphene and producing of new carbon nanomaterials with tunable properties for various applications.


C. elegans hermaphrodite germline, and how this redundancy may contribute to robustness. We focus on the various types of redundancy utilized to regulate this balance, as well as the approaches that have enabled these redundant mechanisms to be uncovered.The proper production of gametes over an extended portion of the life of an organism is essential for a high level of fitness. The balance between germline stem cell (GSC) proliferation  and differentiation (production of gametes) must be tightly regulated to ensure proper gamete production and overall fitness. Therefore, organisms have evolved robust regulatory systems to control this balance. Here we discuss the redundancy in the regulatory system that controls the proliferation vs. differentiation balance in the  C. elegans germline to ensure proper regulation of the balance between germline stem cell (GSC) proliferation and differentiation.Proper development of multicellular organisms requires the ability to adapt to both internal and external perturbations. The ability of an organism to adapt to such changes requires its biological processes to be robust, meaning they are able to continue to develop and function, even when perturbations are encountered . This roStem cells are necessary for proper development and tissue homeostasis; therefore, the processes regulating their behavior must be robust. Stem cells have the ability of maintaining themselves over much of the life of the animal, while also providing the cells that form the differentiated tissue . For steThe reproductive success of animals within a population is key to that population being maintained; therefore, the processes involved in ensuring proper gamete formation must be robust. This includes the balance between proliferation and differentiation of GSCs. If this balance is disrupted, or is unable to withstand various environmental insults, the reproductive success of the animal would be significantly reduced; too much self-renewal would result in germline tumors of proliferating cells and a reduction or elimination of differentiated gametes that are formed, while too much differentiation would result in the GSC population being depleted and a lack of sustained gamete formation.C. elegans germline has revealed a core genetic pathway that contains many redundant factors, and many additional inputs that appear to fine-tune the balance. This regulatory mechanism has been extensively reviewed elsewhere , which is a somatic cell that caps the distal end of the gonad arm and serves as the niche cell for the GSC stem cell population (sygl-1 (synthetic germline proliferation defective) and lst-1  genes (fem-3 mRNA binding factor) and other PUF (Pumilio and FBF) homologues (PROM−1 (skp1-cullin-F-box protein progression of meiosis-1) pathways, based on their founding members  genes  (Kershnet) genes , whose pmologues . This PUmologues . These p members .PROM−1 pathways to increase  of a loss of the redundant effectors, but also the mechanism by which each effector functions. With many of the examples of redundancy in regulating the proliferation vs differentiation balance in the C. elegans germline has proven to be a powerful system to uncover redundant factors and mechanisms due, in part, to the availability of sensitive genetic backgrounds and the ease with which subtle phenotypes can be observed. C. elegans are transparent, allowing observation of the germline in living animals [reviewed in  gain-of-function allele, perhaps the strongest identified glp-1 gain-of-function allele, and which also contain an additional wild-type copy of glp-1 on a free duplication, have completely tumorous germlines with no evidence of cells entering meiosis , all stem cells differentiate prematurely in early larval development, resulting in a total of approximately four to eight stem cells, which differentiate into 16–32 sperm  and glp-1(q175null) result in these extreme opposite phenotypes, there are many other alleles of glp-1, including weak gain-of-function alleles, and partial loss-of-function alleles, which result in milder phenotypes, especially at permissive temperatures (glp-1(oz264gf) or glp-1(ar202gf) at higher temperatures result in increased proliferation of the mitotic cells and a delay of entry into the differentiation pathway, resulting in a larger than wild-type progenitor zone , have stem cells entering the differentiation pathway more distally in the gonad arm, resulting in a smaller progenitor zone than wild-type  as potentially being involved in regulating the proliferation vs. differentiation balance whose involvement have been characterized by analyzing the Glp and tumorous phenotypes that result when combined with weak glp-1 loss and gain-of-function alleles [Reviewed in , function downstream of GLP-1/Notch signaling within the C. elegans germline   . The higof fem-3 . No singfbf RNAi , supportfbf-1 and fbf-2 identified an additional role for fbf in regulating the balance between proliferation and differentiation of GSCs  mutants as the progenitor zone did not become fully depleted until later in larval development, and depletion did not fully occur at higher temperatures , a catalytic subunit of the CCR4-NOT (carbon catabolite repressor factor 4—negative on TATA-less) deadenylation complex   . Localizality 1) . Therefoality 1) . Althougality 1) . These vality 1)  for discality 1) .fbf-1 and fbf-2 within the C. elegans germline suggested that these genes displayed molecular redundancy, with their highly conserved PUF domains allowing them to recognize and repress the same mRNA targets; however, more recent evidence of differing binding partners and localization, combined with their single mutant phenotypes, has uncovered that these two genes likely function using distinct mechanisms to repress target gene expression    works redundantly with fbfs to robustly promote GSC proliferation. PUF-11 was identified as a strong candidate as it displays similar RNA-binding specificity to FBF and interacts with LST-1 in a large scale yeast two-hybrid screen (Although the complete loss of the GSC pool in adult enotype) . In the /animal) . The inad screen . PUF-3, d screen .puf-11 and puf-3, have ∼90% nucleotide identity with each other, and 88% identity at the amino acid level  mutants , based on mRNA in situ data showing expression within the GSC pool  knockdown, resulted in a Glp phenotype during larval development  normally and result in over-proliferation of the mitotic cells, with most cells failing to enter meiosis . Its placement in this pathway is based on the finding that gld-2; nos-3 double mutants show synthetic tumors while gld-1; nos-3 double mutants have normal meiotic entry )  that functions in parallel to the GLD-1 and GLD-2 pathways to inhibit proliferation and/or promote meiotic entry. First, the synthetic tumorous germlines in ic cells . Second,p-1(gf)) . Finally mutants . Therefoic entry .PROM−1 was found to bind CYE-1, potentially targeting it for degradation  that functions redundantly with the GLD-1, GLD-2 and SCFPROM−1 pathways to promote meiotic entry and/or inhibit proliferation  E3 ubiquitin-ligase complex which includes the PROM-1 F-box protein. SCFradation . Loss ofpathways . Further mutants . Howeverferation .PROM−1 pathways display cellular process redundancy in their regulation of the proliferation vs differentiation decision in the C. elegans germline ) and prp-17 (Yeast PRP17 related splicing factor) were identified , cyn-4/mog-6 and prp-19  mutants , which encodes an ⍺-subunit of the 20S proteasome  was found to be one of these proliferation promoting proteins, and is targeted for degradation by the E3 ubiquitin ligase RFP-1 (ring finger protein) (glp-1(gf) background, suggesting that there could be multiple targets whose regulated degradation is necessary for the proliferation vs differentiation decision to be properly maintained  . Other fintained . As mentintained .glp-1 partial loss-of-function alleles, or enhance the tumorous phenotype of glp-1 gain-of-function alleles [reviewed in , which could impact over-proliferation and under-proliferation phenotypes. Additionally, the type and degree of redundancy may not be constant throughout development and ageing. For example, it is possible that some factors are redundant in early larval development as the progenitor zone is being established, but not in adulthood while it is being maintained.The genetic pathway regulating the proliferation vs. differentiation decision in the Presumably, even slight disruptions in the proliferation vs. differentiation balance would negatively impact long-term reproductive success. Therefore, there is likely significant evolutionary pressure ensuring that the system regulating the proliferation vs differentiation balance is robust, which could involve a significant level of redundancy. It is thought that redundancy can provide robustness to a system by decreasing the phenotypic consequence of mutations, or by broadening the flexibility of the system so that signals can still be properly controlled in fluctuating cellular environments, different genetic backgrounds, and environmental stresses such as changes in temperature and availability of food . TherefoC. elegans germline. In some cases, the initial discovery of redundant factors and their presence or lack of similarity suggested that they were an example of one class of redundancy. However, for some redundant factors, as understanding increased as to their modes of action and their potential targets, it actually became less clear as to which class of redundancy most accurately described their relationship. Furthermore, some redundant factors may exist in more than one class; for example, two redundant proteins with a high degree of amino acid sequence identity may function on a group of common targets, but may also have targets that are unique to each redundant factor. Conclusive classification of redundant factors requires complete understanding of the biochemical functions of the factors and a knowledge of all their targets. Obviously, we are not yet at that level of understanding with respect to the regulation of the proliferation vs differentiation decision in the C. elegans germline. However, perhaps our level of understanding of the biochemical functions and targets involved in this process is relatively high, which has revealed that the redundant relationships between factors are more complex than previously appreciated. Indeed, the relationships between redundant factors in other systems may also be found to be more complex than once thought as more is learned about the biochemical functions and targets of these redundant factors. However, what is clear is that many types of redundancy are utilized in regulating this proliferation vs. differentiation balance in the C. elegans germline, and that this redundancy likely contributes to its reproductive success and fitness.In this review we have discussed some of the different types of redundancy, highlighted with specific examples, that are utilized in regulating the proliferation vs differentiation decision in the


Participants in a conversation must carefully monitor the turn-management (speaking and listening) willingness of other conversational partners and adjust their turn-changing behaviors accordingly to have smooth conversation. Many studies have focused on developing actual turn-changing  models that can predict whether turn-keeping or turn-changing will occur. Participants' verbal and non-verbal behaviors have been used as input features for predictive models. To the best of our knowledge, these studies only model the relationship between participant behavior and turn-changing. Thus, there is no model that takes into account participants' willingness to acquire a turn (turn-management willingness). In this paper, we address the challenge of building such models to predict the willingness of both speakers and listeners. Firstly, we find that dissonance exists between willingness and actual turn-changing. Secondly, we propose predictive models that are based on trimodal inputs, including acoustic, linguistic, and visual cues distilled from conversations. Additionally, we study the impact of modeling willingness to help improve the task of turn-changing prediction. To do so, we introduce a dyadic conversation corpus with annotated scores of speaker/listener turn-management willingness. Our results show that using all three modalities  of the speaker and listener is critically important for predicting turn-management willingness. Furthermore, explicitly adding willingness as a prediction task improves the performance of turn-changing prediction. Moreover, turn-management willingness prediction becomes more accurate when this joint prediction of turn-management willingness and turn-changing is performed by using multi-task learning techniques. Turn-changing is an important aspect of smooth conversation, where the roles of speaker and listener change during conversation. For smooth turn-changing, conversation participants must carefully monitor the speaking and listening (turn-management) willingness of other conversational partners and consider whether to speak or yield on the basis of their own willingness and that of their partner.To realize a dialogue system that can interact as smoothly as humans do, the dialogue system must be able to switch between listening and speaking at appropriate times, just as humans do. Therefore, predicting turn-changing can be helpful for conversational agents or robots as they need to know when to speak and take turns at the appropriate time. The field of human-computer interaction has long been dedicated to computational modeling of turn-changing. Many studies have focused on developing actual turn-changing  models that can predict whether turn-keeping or turn-changing will happen using participants' verbal and non-verbal behaviors  Is actual turn-changing taking place in accordance with the participant's willingness to speak? In other words, is there consonance between turn-management willingness and actual turn-changing?Q2-1) Are the verbal and non-verbal behaviors of speakers and listeners useful in predicting turn-management willingness?Q2-2) Does explicitly modeling willingness help with turn-changing prediction?First, we analyze the relationship between turn-management willingness and actual turn-changing in our empirical study to address Q1. In particular, we investigate the relationship between the actual turn-changing and turn-management willingness of conversation participants to determine whether turn-changing occurs in accordance with the turn-management willingness in dialogue.Second, we study the relative behavioral usefulness of features obtained from acoustic, linguistic, and visual modalities, from both speakers and listeners, to address Q2-1. Predicting this willingness directly has the potential to support conversational agents and robots with appropriate starting and ending utterances to have smooth conversations. To respond to Q2-2, we built predictive models for actual turn-changing prediction. As a first step, we use trimodal inputs, i.e., acoustic, linguistic, and visual cues, to directly predict turn-changing. As a second step, we incorporate willingness prediction into turn-changing prediction. This joint modeling approach is motivated by the intuition that humans are likely to control actual turn-changing on the basis of speaking and listening willingness. We build a multi-task model for joint turn-changing and willingness prediction, and then we evaluate the effectiveness of our proposed approach in terms of performance improvement. To answer these research questions, we collected a new dyadic dialogue corpus including audio and video recordings, and we transcribed the content of dyad conversations as well as acquired annotations of speaking and listening willingness scores for both conversation participants. The dataset was collected to empirically study the various combinations of speaking and listening willingness with turn-changing-based multimodal behavioral markers extracted from these three modalities . As the method for collecting willingness scores, it is very difficult for participants to evaluate their own internal state in dialogue. We deal with scores of willingness as perceived by several third parties observing the participants.In Section 2, we review relevant related work and highlight our motivation to propose new approaches to turn-management willingness. Section 3 describes the corpus data collection using dyad interactions. Section 4 describes our analysis of turn-management willingness and actual turn-changing to address Q1. Sections 5 and 6 describe the implementation and evaluation of the proposed predictive models for turn-willingness and turn-changing to address Q2-1 and Q2-2. Section 7 discusses the results for Q1, Q2-1, and Q2-2. We conclude in Section 8 with a brief summary and mention of our future work.Smooth turn-changing is an important aspect of social communication. It is one of the main means of coordinating one's communicative actions and interacting in a successful manner with others. In general, a turn refers to an interlocutors' right to speak, or “hold the floor,” while the partner (or partners) listens Kendon, . A turn To elaborate, the mutual interaction of gaze behavior is thought to contribute to smooth turn-changing  regardless of whether or not the human with whom it is interacting wants to speak or not. For example, in Schlangen  and AtteAlso, using our estimation of turn-management willingness, agents may be able to facilitate users' in speaking on the basis of the users' willingness. For example, although a listener may strongly want to take a turn, he/she may not actually be able to do so . At such moments, the agent may be able to prompt the listener to start speaking using verbal and non-verbal behavior .We collected a new corpus (named the “MM-TMW Corpus”) that contains verbal and non-verbal behavioral information on human-human dialogues. It consists of 12 face-to-face conversations of people who had never met before (12 groups of 2 people). The participants were 24 Japanese  in their 20–50 s . Participants were recruited from the general public at large through a staffing agency. The age difference between pairs and the number of male-female pairs were set to be as varied as possible. All participants gave informed consent. They were seated in close proximity to each other in a quiet environment. The conversations were structured to be about multiple topics, including taxes and social welfare balance. The lengths of these conversations were unified to be around 10 min. The total time of all conversations was 120 min. The participants' voices were recorded by a headset microphone. The entire discussions were recorded by a camera. We also took upper body videos of each participant recorded at 30 Hz. A professional transcribed all Japanese utterances, and another double-checked the transcripts.As a first step, professional annotators identified the spoken utterance segments using the annotation scheme of the inter-pausal unit (IPU)  to the point of one frame  before the beginning of the next utterance to annotate willingness scores see . The annTurn-holding willingness (speaker's willingness to speak): Does the speaker have the will to hold the turn (continue speaking)?Turn-yielding willingness (speaker's willingness to listen): Does the speaker have the will to yield the turn (listen to listener's speaking)?Turn-grabbing willingness (listener's willingness to speak): Does the listener have the will to take a turn (start speaking)?Listening willingness (listener's willingness to listen): Does the listener have the will to continue listening to the speaker?0.870: ICC = 0.904 for speaker's willingness to speak, ICC = 0.877 for speaker's willingness to listen, ICC = 0.878 for listener's willingness to speak, and ICC = 0.875 for listener's willingness to listen. A high annotation agreement measured by ICC suggests that the data was very reliable. We used the average values of the 10 annotators as willingness scores.The annotators scored each willingness index on a 5-point Likert scale, where 1 meant “He/she is not showing willingness,” 5 meant “He/she is showing strong willingness,” and 3 meant “uncertain.” We had the 10 annotators score all videos to ensure good reliability. We calculated the rater agreement using the Intraclass Correlation Coefficient (ICC). The ICC scores for all four categories were over In this section, we analyze the relationship between willingness scores and actual turn-changing or turn-keeping as an empirical study. the speaker prefers to continue speaking, and the listener prefers to continue listening” during turn-keeping. Corresponding t-tests were conducted to determine if there was a statistically significant difference between turn-holding and turn-yielding willingness scores. The results showed that there was a significant difference between turn-holding and turn-yielding scores . Similarly, there was a significant difference between scores for turn-grabbing and listening willingness . This result is not too surprising, and it suggests that the participants' willingness and turn-keeping were consistent .This means that the person who becomes the next speaker has a high speaking willingness and the person who becomes the next listener has a high listening willingness in turn-keeping. In other words, “t-test showed that there was no significant difference between turn-holding and turn-yielding scores and between turn-grabbing and listening willingness scores.For turn-changing, all the average willingness scores were from 3.0 to 3.5, with larger standard deviations. This result suggests that around turn-changing, the person who becomes the next speaker does not always have a high speaking willingness, and the person who becomes the next listener does not always have a high listening willingness. The results of a corresponding Therefore, the relationship between willingness and turn-changing is more complex than expected. There may be cases of dissonance between willingness and actual turn-changing as answer to research question Q1.rSS, rSL, rLS, rLL) to represent the ordering position among all willingness scores, where rSS, rSL, rLS, rLL are the ranks of the speaker's speaking, speaker's listening, listener's speaking, and listener's listening willingness, respectively. For example, rSS = 1 if the speaker's speaking willingness score has the highest value among all of the scores. As another example, when the scores for speaker's speaking, speaker's listening, listener's speaking, and listener's listening willingness were 3.57, 2.95, 1.82, and 4.44, the corresponding pattern was denoted as .To study this relationship in detail, we analyzed the relative ordering of speaking and listening willingness scores when co-occurring with turn-keeping or turn-changing. We denote willingness combinatorial patterns as (*. We discuss these results in the following paragraphs.the speaker prefers to continue speaking” (rSS<rSL) or “the listener prefers to continue listening” (rLS>rLL). As shown in 96.4% for turn-keeping, which suggests that there were few cases of dissonance between willingness and turn-keeping. For turn-changing, consonance cases happen when “the speaker prefers to start listening” (rSS<rSL) or “the listener prefers to start speaking” (rLS<rLL). Consonance patterns took up 49.6% for turn-changing. This means that consonance cases almost always happen during turn-keeping but not during turn-changing.Consonance cases happen during turn-keeping when “the speaker wants to listen, and the listener wants to speak” (rSS>rSL and rLS<rLL). This means that the preferences of both the speaker and listener are in harmony. Thus, a turn-change should occur. For turn-changing, dissonance cases happen when “the speaker wants to speak, and the listener wants to listen” (rSS<rSL and rLS>rLL). In this case, both participants want to keep their role. In *. As shown in the figure, 3.6% of cases of dissonance occurred for turn-keeping, while 50.4% were for turn-changing. The large number of cases between willingness scores and turn-changing indicates the importance of estimating the willingness score. The next speaker decided against his or her own willingness as well as their partner's willingness. Such occurrences have the potential to frustrate participants during a conversation.During turn-keeping, dissonance cases happen when “The analysis results in Section 4 suggest that the willingness scores sometimes had discrepancies with actual turn-changing. The accuracy could be further improved by performing multi-task learning on willingness and turn-changing since they have a strong relationship. Multi-task learning is the process of learning latent representations that are shared among multiple related tasks. In the field of deep learning, deep learning-based approaches in which the parameters of hidden layers are shared by multiple tasks are well-established and commonly used. Learning multiple related tasks simultaneously has been demonstrated to improve the prediction performance of each single task while exploiting commonalities and differences across tasks Ruder, . For thaQ2-1, we implemented three kinds of models for predicting turn-management willingness by using the multimodal behaviors of either the speaker or listener or both of them. By investigating and comparing the performance of the models, we demonstrate that turn-management willingness can be predicted by using the multimodal behaviors of speakers and listeners. To address Q2-2, we also implemented models for predicting turn-changing that jointly predict turn-management willingness on the basis of single turn-management predictive models. By comparing the performance of the models between using multi-task learning and single-task learning, we demonstrate that joint-willingness prediction can improve the performance of turn-changing prediction.To address We used the features of behaviors extracted during IPUs  as input for the predictive models the same as other research on turn-changing prediction  can be trained in a hidden layer and extracted for transition learning   and classification models (for turn-changing/keeping prediction), respectively. A multi-task model was then learned to jointly predict willingness and turn-changing/keeping. This helps to understand the impact of modeling willingness on turn-changing explicitly. The architecture of our model is illustrated in We formulated turn-management willingness prediction as a regression task and average willingness scores from the 10 annotators as the ground truth. We trained deep neural network-based predictive models to estimate willingness scores. The unimodal features were first fed into individual processing modules to be further processed as 64-dimensional embeddings. For acoustic and visual modalities, the processing module was a one-hidden-layer gated recurrent unit (GRU)  was used as the loss function.To embed knowledge on willingness into turn prediction, our proposed multi-task model jointly predicts willingness scores and turn-changing/keeping. In particular, the turn-management willingness and turn-changing prediction tasks are simultaneously learned by the proposed multi-task predictive model. The architecture of the proposed model is designed on the basis of the neural architecture discussed above. The main difference is that, after the fusion layer, there is a FC layer for each task. The entire loss function is a weighted average of MSE and CE with weights of 1 and 2, respectively.Q2-1, we implemented different predictive models for turn-management willingness prediction based on the multimodal behaviors of either the speaker or listener or both. We investigated and compared the performance of the models to demonstrate that turn-management willingness can be predicted by using the multimodal behaviors of speakers and listeners. To answer Q2-2, we also implemented models for turn-changing prediction that jointly predict turn-management willingness and turn-changing. We compared the performance of the multi-task learning models and single-task models to demonstrate that incorporating willingness into turn-changing predictive models improves turn-changing prediction.To answer All models were trained using the Adam  between predicted and actual scores . A high CCC value indicates high agreement between the values of the predicted scores and ground truth. This means that prediction and ground truth values are similar to each other, and the general trend changes for both signals are the same  to (7) were predictive models fed with acoustic, linguistic, or visual modalities of speaker or listeners, respectively. The performance of these models was evaluated to verify if each modality was effective and discriminative for our prediction tasks. Models (8) to (10) were predictive models that were fed with a single modality of speakers and listeners. In detail, model (8) was applied to speech features, while model (9) was fed with linguistic features. Furthermore, model (10) was trained on visual features. By comparing models (8) to (10) with models (2) to (7), we could verify whether using the features of speakers and listeners is more effective than using the features of either speaker or listener. Models (11) and (12) were predictive models that used features from the three modalities of speakers or listeners. Model (13) was a predictive model that learns on input features extracted from the three modalities of both the speaker and the listener. By comparing models (11) to (13) with models (2) to (7), we could examine the usefulness and discriminative power of multimodal features over features extracted from a single modality. These comparisons were made to address Q2-1. Model (14) was a multi-task learning predictive model trained on the three modalities of speakers and listeners to simultaneously predict turn-management willingness and turn-changing. This model (14) was compared with model (13), which does not use multi-task learning techniques. These comparisons were intended to respond to Q2-2.p < 0.05 or p < 0.001). This suggests that a listener's acoustic features are most useful for predicting speaker and listener turn-management willingness among the unimodal features of speakers and listeners.First, we examined which of the speaker's and listener's unimodal features was most useful for estimating turn-management willingness when used alone. As shown in p < 0.001). We compared model (9) with models (3) and (6) to demonstrate that linguistic features from both the speaker and listener are more useful than using only those from either the speaker or listener. The CCCs for model (9), 0.411, 0.409, 0.434, and 0.403, were significantly higher than those of models (3) and (6) (p < 0.001). We compared model (10) with models (4) and (7) to demonstrate that linguistic features from both the speaker and listener are more useful than using only those from either the speaker or listener. The CCCs for model (8), 0.201, 0.152, 0.149, and 0.164, were significantly higher than those of models (4) and (7) except for the case of turn-grabbing willingness between models (4) and (10) (p < 0.05 of p < 0.001). These suggest that using unimodal features from both the speaker and listener to estimate turn-management willingness was more useful than using them alone.We examined whether using unimodal features from both speakers and listeners to estimate turn-management willingness is more useful than using those from one of them alone. As shown in p < 0.001). This suggests that acoustic features are most useful for turn-management prediction when using unimodal features from both the speaker and listener.Comparing models (8) to (10), the CCCs of turn-management willingness prediction for model (10) were significantly highest . In contrast, the CCCs of turn-grabbing and listening prediction for model (12), 0.485 and 0.465, were significantly higher than the CCCs for model (11), 0.344 and 0.395 (p < 0.001). This suggests that speaker/listener features are more useful for predicting speaker/listener turn-management willingness than listener/speaker willingness.Comparing models (11) and (12), the CCCs of turn-holding and turn-yielding prediction for model (11), 0.497 and 0.425, were significantly higher than those of model (12), 412 and 0.376 (p < 0.001). This suggests that a model using features from both speakers and listeners outperforms a model using them from one person. Comparing model (5), which had the highest CCC when using only unimodal features, and (13), which had the highest CCC when using multimodal features, the CCCs of turn-holding and turn-yielding prediction for model (13) were significantly higher than that of model (5) (p < 0.001). We found an overall improvement in turn-management willingness prediction by fusing the multiple features of speakers and listeners.Comparing model (13) with models (11) and (12), model (13) with all features performed best, 0.556 for turn-holding, 0.504 for turn-yielding, 0.573 for turn-grabbing, and 0.549 for listening, being significantly higher than models with speaker features (11) or listener features (12)  to (13) similarly to the turn-management predictive models to assess the effect of multi-task learning on turn-changing prediction. We report the performance of the models to confirm whether our extracted speaker and listener features were useful for turn-changing prediction.p < 0.05). This suggests that a speaker's linguistic features are most useful for predicting turn-changing among the unimodal features of speakers and listeners.First, we examined which of the speaker's and listener's unimodal features was most useful for predicting turn-changing when used alone. Comparing the performances of the models among (2) to (7), the F1 score for model (3), 0.735, was significantly higher than that of the model that had the second highest values, 0.715 for (5) (p < 0.05). We compared model (9) with models (3) and (6) to demonstrate that linguistic features from both the speaker and listener are more useful than using only those from either the speaker or listener. The F1 score for model (9), 0.745, was significantly higher than that of model (6) (p < 0.05). We compared model (10) with models (4) and (7) to demonstrate that linguistic features from both the speaker and listener are more useful than using only those from either the speaker or listener. The F1 score for model (10), 0.675, was significantly higher than those of models (4) and (7) (p < 0.05 and p < 0.001). These suggest that using unimodal features from both the speaker and listener for estimating turn-changing was more useful than using only those from one of them.We examined whether using unimodal features from both the speaker and listener to predict turn-changing is more useful than using them alone. We compared model (8) with models (2) and (5) to demonstrate that acoustic features from both the speaker and listener are more useful than using only those from either the speaker or listener. The F1 score for model (8), 0.720, was significantly higher than that of model (2) (p < 0.001). This suggests that acoustic features are most useful for turn-management prediction when using unimodal features from both the speaker and listener.Comparing models (8) to (10), the CCCs of turn-management willingness prediction for model (9) were significantly highest . This suggests that the speaker's features are more useful for predicting turn-changing.Comparing models (11) and (12), the F1 score of turn-changing prediction for model (11), 0.759, was significantly higher than that of model (12), 711 (p < 0.001). This suggests that the model using the features from speaker and listener more effective than the model only using the features of speaker or listener. We found an overall improvement in turn-changing prediction by fusing multiple speaker and listener features. These results are in line with previous research that similarly used both speaker and listener behaviors for turn-changing prediction  with (12), model (13) with all features performed best, 0.771, which was significantly higher than model (12) ] even though the prediction task is known to be difficult and our dataset is relatively small. As an alternative, features from pre-training models such as VGGish, BERT, and ResNet-50 could be used to mitigate our relatively small dataset. Turn-changing predictive models (2) to (4) can serve as a baseline for evaluating the effect of using multi-task learning.p < 0.001). This suggests that multi-task learning incorporating turn-management willingness prediction into turn-changing predictive models improves the performance of turn-changing prediction.We first analyzed whether applying multi-task learning to turn-management willingness and turn-changing prediction can improve turn-changing prediction. Model (14) used multi-task learning in addition to model (13). We compared the performance between models (13) and (14) for turn-changing prediction. Model (14) had a significantly higher F1 score, 0.797, than model (13), 0.771 (p < 0.05). This suggests that multi-task learning also improved the performance of the speaker's turn-management willingness prediction only when using the features of speakers and listeners.We also analyzed whether multi-task learning is useful for predicting turn-management willingness. We compared the performance between models (13) and (14). Model (14) had significantly higher CCCs, 0.580 for turn-holding, 0.530 for turn-yielding, and 0.560 for listening, than model (13), 0.556 for turn-holding, 0.504 for turn-yielding, and 0.549 for listening . This dissonance between willingness and actual speaking behavior during turn-changing is the first time that such a dissonance has been revealed. Previous studies have focused primarily on what verbal and non-verbal behaviors humans engage in during turn-changing. In the future, it may be possible to examine how verbal and non-verbal behaviors change depending on the type of participant's willingness during turn-changing.The results also suggest that there is a possibility that willingness prediction could be beneficial for building an agent that has smooth turn-management based on the discrepancies between willingness and actual turn-changing. For example, the agent may be able to prompt a listener to take a turn and start speaking by exhibiting verbal and nonverbal behavior.The willingness we collected was not reported by the individuals participating in the dialogue but by a third party observing the video. The agreement rate of the willingness data annotated by the 10 outside observers was very high and considered to be of high quality. However, it is not certain that judgments on willingness by third parties and the participants themselves will be exactly the same. It would be desirable to examine what differences exist when participants themselves and third parties annotate willingness. The data we used was dialogue data between two Japanese in a discussion featuring divergent opinions. It is conceivable that the topic, number of people, culture, language, and other conditions could have a variety of effects on speaker alternation. It would also be interesting to examine how these conditions affect the relationship between willingness and actual turn-changing.Our results show that the listener's acoustic information is most useful for predicting the speaking and listening willingness of speakers and listeners when using only a single feature from speakers or listeners. Generally, the listener uses short bits of speech and non-verbal behaviors such as nodding, changing the head direction, and gazing Duncan, . Our resMoreover, our results show that the features of both speakers and listeners are useful for predicting turn-management willingness. Individual turn-management willingness can be predicted better using features from individuals rather than from others. Individual willingness is well reflected in an individual's behavior. Moreover, the models using features of both speakers and listeners performed better than those using only speaker or listener features. When using features of both the speaker and listener, the acoustic modality is the best-performing modality for predicting the speaking and listening willingness among the trimodal features. One explanation is that the listener's acoustic information is most useful for predicting speaking willingness when using a single feature from a single modality. Moreover, the multimodal approach with the trimodal features of the speaker and listener is most useful in predicting the turn-management willingness of both persons. In other words, the turn-management willingness of a speaker and listener can influence the verbal and non-verbal behaviors of both. This suggests that predicting the internal state of an individual, such as their willingness, using features from not only the individual but also conversational partners could be greatly useful in dyad interactions.Turn-changing prediction becomes most accurate when turn-management willingness and turn-changing are predicted simultaneously using multi-task learning. This demonstrates that explicitly adding willingness as a prediction target improves the performance of turn-changing prediction. This introduces new possibilities for more accurately predicting human behavior by predicting human psychological states at the same time in conversations. Moreover, models that jointly learn two tasks also improve the performance of turn-management willingness compared with models that perform just one task. The multi-task learning approach allows a model to learn the underlying relationship between willingness scores and turn-changing. This results in both improved turn-changing and turn-management willingness prediction. These results also suggest that a multi-task prediction approach that predicts the internal state of people, such as their willingness and actual behaviors, could be greatly useful in dyad interactions. Applying such an approach to tasks other than turn-changing prediction will be part of our further investigation.For conversational agents or robots to start or stop speaking at the right time, we do believe that predicting human turn-management willingness is critically important, rather than simply predicting the next speaker . In this study, we attempted to predict the willingness of two conversation partners simultaneously during dyad interactions. When considering a human-agent interaction (HAI) scenario, our approach would need to be adapted to predict only one user's willingness using the tri-modal features, for either a speaker role or a listener role. This is one of our future research directions.Modeling turn-management willingness may aid in detecting discrepancies between willingness toward turn-changing and actual turn-changing. A conversational system can then recognize users having a high willingness to speak (speaker's turn-holding or listener's turn-grabbing willingness) even though they cannot speak. It could even help to mediate meetings by possibly interrupting the current speaker if a person does not notice that the conversation partner has a low willingness to listen. Many studies have been conducted to facilitate human interactions with agents and robots. For example, robots have been proposed to prompt the user who has the least dominance in a conversation . Conversational agents could perform smooth turn-changing and facilitate users in speaking with the prediction results of turn-management willingness and actual turn-changing. We built models for predicting the turn-management willingness of speakers and listeners as well as turn-changing with trimodal behaviors, acoustic, linguistic, and visual cues, in conversations. An evaluation of our models showed that turn-management willingness and turn-changing are predicted most precisely when all of the modalities from speakers and listeners are used. Furthermore, turn-changing prediction becomes more accurate when turn-management willingness and turn-changing are predicted jointly using multi-task learning. Turn-management willingness prediction also becomes more accurate with combined prediction. These results suggest that more accurate predictive models of human behaviors could be built by incorporating other predictions related to human psychological states.The original contributions presented in the study are included in the article/supplementary files, further inquiries can be directed to the corresponding authors.The studies involving human participants were reviewed and approved by the Ethical Review Committee at NTT Corporation. The patients/participants provided their written informed consent to participate in this study.RI and L-PM contributed to conception and design of the study. RI organized the corpus and wrote the first draft of the manuscript. XR performed implementation and experiment of prediction model. XR and MM performed the statistical analysis. RI, XR, MM, and L-PM wrote sections of the manuscript. All authors contributed to manuscript revision, read, and approved the submitted version.XR and L-PM were partially supported by the National Science Foundation (#1722822 and #1734868). MM was supported by the Swiss National Science Foundation (#P2GEP2_184518). RI was supported by the NTT Corporation. The funder was not involved in the study design, collection, analysis, interpretation of data, the writing of this article or the decision to submit it for publication.Author RI was employed by NTT Corporation. The author was a visiting researcher at CMU and conducted this research as a research fellow at CMU. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.