We introduce high-performance metal mesh/graphene hybrid transparent conductive layers (TCLs) using prime-location and metal-doped graphene in near-ultraviolet light-emitting diodes (NUV LEDs). Despite the transparency and sheet resistance values being similar for hybrid TCLs, there were huge differences in the NUV LEDs’ electrical and optical properties depending on the location of the graphene layer. We achieved better physical stability and current spreading when the graphene layer was located beneath the metal mesh, in direct contact with the p-GaN layer. We further improved the contact properties by adding a very thin Au mesh between the thick Ag mesh and the graphene layer to produce a dual-layered metal mesh. The Au mesh effectively doped the graphene layer to create a p-type electrode. Using Raman spectra, work function variations, and the transfer length method (TLM), we verified the effect of doping the graphene layer after depositing a very thin metal layer on the graphene layers. From our results, we suggest that the nature of the contact is an important criterion for improving the electrical and optical performance of hybrid TCLs, and the method of doping graphene layers provides new opportunities for solving contact issues in other semiconductor devices. Graphene itself has a high value of sheet resistance (300–1500 Ω/□) resulting from imperfect growth and is therefore inappropriate for use in TCLs5. There have been multiple efforts to modify graphene using metal nanoparticles, metal nanowires, and a metal mesh to achieve the necessary performance characteristics and make graphene a suitable replacement for the oxide-based TCLs in UV LEDs and flexible devices12. However, despite reports of excellent performance for modified graphene, graphene/metal nanoparticle hybrid TCLs still exhibit unacceptable levels of sheet resistance caused by the discrete nanoparticles’ structures6. Moreover, even when the graphene/metal nanowire hybrid TCLs exhibit proper values for their sheet resistance (10–100 Ω/□) and transmittance (~85%), it has been difficult to control the uniformity of the thin nanowire films precisely and ensure their reliable contact characteristics in optoelectronic devices12. These drawbacks have led to the degradation of electrical contact properties in attempts to fabricate practical optoelectronic devices.Graphene-based transparent conductive layers (TCLs) have been attracting attention as alternatives to conventional TCLs in optoelectronic devices, such as solar cells and light-emitting diodes (LEDs). They offer several advantages over existing oxide-based TCLs, including (i) high transparency without wavelength dependency, (ii) high conductivity, and iii) good flexibilityAmong the various candidates, graphene/metal mesh hybrid TCLs are noteworthy for their use in practical applications because they have a low value of sheet resistance (1–100 Ω/□) and high transparency (~90%). Furthermore, the electrical and optical properties of hybrid TCLs can be sufficiently controlled by rationally designing the features of the metal mesh, such as the gaps between lines, line widths, and thicknesses.10. We assume that the undesirable results for the hybrid TCLs in optoelectronic devices can be attributed to several issues: (i) the structure of the metal mesh, (ii) the optimum position of each material due to its hybridization, and (iii) the contact resistance of TCLs in devices.Despite these advantages, there are presently few research results reporting the development of graphene/metal mesh hybrid TCLs for optoelectronic devices, and performance reported for both the electrical and optical properties of hybrid TCLs have fallen short of expectations13. However, achieving high-performance conditions for actual optoelectronic devices still requires determining the optimum position of each material and improving its contact properties with the p-GaN layer.We have previously reported the successful design of a metal mesh, and solved the structural issue in GaN-based near ultra-violet light-emitting diodes (NUV LEDs) by adding a supporting layer in the metal mesh to guarantee its stable contact properties13. We observed huge differences in the electrical and optical properties depending on the location of the graphene in the NUV LEDs even if each TCL showed similar sheet resistance and transmittance values, regardless of the graphene’s location. A main factor in these differences was the change made in the contact with the p-GaN layer, accomplished by changing the location of the graphene. Furthermore, we adopted a dual-layered metal mesh  to dope the graphene layer. Giovannetti et al. first reported the effect of doping a graphene layer by depositing metal, and several groups have since reported on the principles involved in the doping effects, both theoretically and experimentally15. However, to date, there have been few results involving practical devices. In our work, we have obtained improvements for the contact properties in the NUV LEDs by doping the graphene layer with a 5 nm-thick Au/150 nm-thick Ag mesh, and we have verified the doping effect through Raman spectra, work function variations, and transfer length methods (TLMs). The superior changes in the graphene layer using this simple metal deposition technique provide many opportunities to control contact properties in other semiconductor devices.In this work, we demonstrate performance when a graphene layer is located either above or below the metal mesh, the doping of the graphene layer by incorporating a dual-layered metal mesh, and their effect on the performance of GaN-based NUV LEDs. Thanks to the high transparency of the hybrid films in the UV region, we were able to obtain remarkable optical properties in GaN-based NUV LEDs compared to conventional ITO. We used the metal mesh structure as reported in our previous work Even when the metal mesh and/or hybrid films exhibited similar sheet resistances and transmittance values, significant differences were observed in the electrical and optical performances of the NUV LEDs depending on the existence and location of the graphene (prime-location graphene).We were able to dope the graphene layers by depositing a dual-layered metal mesh on them; this improved the contact resistance for the p-GaN layer . Since NUV LEDs fabricated with a 150 nm-thick Ag mesh combined with graphene could act as both prime-location graphene and metal-doped graphene, we designated them “Mesh on GR/Ag doping”.Figure As can be seen in Fig. Furthermore, the light output power of the NUV LEDs exhibited huge variations depending on the type of very thin metal mesh layer on the graphene layer. Specifically, while NUV LEDs with “Ni doping” exhibited the lowest light output power among all samples, we obtained the highest light output power in the NUV LEDs with “Au doping”. Figure 16. Even though the sheet resistance values of the “ITO” and “GR” TCLs were high , those of the other TCLs with the Ag mesh structures were very low: 4.88 Ω/□ for “Mesh”, 5.39 Ω/□ for “GR on Mesh”, and 4.54 Ω/□ for “Mesh on GR”. We were able to observe differences in the sheet resistance values based on the existence and location of the graphene layer. The lower sheet resistance values of “Mesh on GR” than “Mesh” were attributed to the graphene layer bridging the gaps of the Ag mesh. However, the higher sheet resistance values of “GR on Mesh” than “Mesh” were due mostly to the torn and rolled-up condition of the graphene layers, even though the graphene layers connected the Ag mesh, as shown in Fig. 17. Next, we investigated the Raman spectra  to identify the graphene layers above/below the Ag mesh. In Fig. −1 and 1586 cm−1) and 2D peaks (2694 cm−1 and 2683 cm−1), respectively. Notably, there were peak shifts in the G and 2D peaks in “4” on the graphene layer on the Ag mesh. These shifts are further discussed in the section on metal-doped graphene. In contrast, the G peak (1591 cm−1) and 2D peak (2693 cm−1) appeared only in a gap region in the “Mesh on GR” TCL. To investigate the contact properties between the TCLs and p-GaN layers depending on the location of the graphene layers, we fabricated TLM samples using pure p-GaN layers. The process of preparing the TLM samples is shown in the inset of Fig. 2 for “ITO”, 0.53 Ω·cm2 for “Ag”, 0.81 Ω·cm2 for “GR on Ag”, and 1.23 Ω·cm2 for “Ag on GR” , a graphene layer (“GR”), a 150 nm-thick Ag mesh (“Mesh”), a graphene layer on a 150 nm-thick Ag mesh (“GR on Mesh”), and a 150 nm-thick Ag mesh on a graphene layer (“Mesh on GR”) on double-side-polished sapphire (DSPS) substrates to investigate the transparency and sheet resistance. We measured the transmittance values in the range 350–600 nm with a UV-VIS-IR spectrometer , as shown in Fig. 21. However, the chemical doping of graphene has several disadvantages, such as instability in external environments. For that reason, in this work we doped the graphene by simply depositing a very thin metal layer. A number of researchers have previously reported the potential doping of graphene using thin metal layers22. By investigating with Raman analysis, the measurement of work function, and TLMs, we determined the effect of doping the graphene using very thin metal deposition, and we describe how to incorporate their advantages in our devices below.Even though the remarkable transparency and sheet resistance of the TCLs was established, some uncertainty remains regarding device performance. For example, we previously observed that there were huge variations depending on the location of the graphene layer, even when they had similar transparency and sheet resistance values. Accordingly, the contact issues require careful consideration to obtain high-performance TCLs. To improve contact properties when using graphene-based TCLs, the graphene is typically doped using chemical methods24. In order to verify the doping effect of the very thin metal layer on the graphene layer, we prepared samples for Raman analysis, as can be seen in Fig. Raman spectroscopy is one of the most powerful tools for investigating changes in the graphene layer. Many research groups have used Raman spectroscopy to characterize chemically doped graphenepz states with metal d states24. This overlap of the band structure results in electron diffusion from the Ni layer to the graphene layer by repulsive force of the electrons caused by Pauli exclusion principle and causes “Ni doping” n-type doping. Thus, there are two types of adsorption for metal-graphene: physisorption  and chemisorption . It is necessary to use materials with large work functions for the p-GaN layer to form better contact properties. In Fig. In addition to using Raman spectra to verify doping type, we checked the doping effect of graphene by measuring the work function of the graphene with the very thin metal layer using ultraviolet photoelectron spectroscopy. Figure 2 for “Ag doping”, 1.1 Ω·cm2 for “Ni doping”, and 0.08 Ω·cm2 for “Au doping”  for mesa masks. The mesa structures were formed through photolithography with positive photoresist (PR) (AZ 4330) and inductively coupled plasma (ICP) etching, as can be seen in Fig. 2 layers using acetone and an HF solution sequentially. Using the mesa structures of the NUV LEDs, we then fabricated six types of TCLs on the p-GaN layers: 150-nm-thick ITO (“ITO”), 150-nm-thick Ag mesh (“Mesh”), a graphene layer on 150-nm-thick Ag mesh (“GR on Mesh”), 150-nm-thick Ag mesh on a graphene layer (“Mesh on GR/Ag doping”), 150-nm-thick Ag mesh and 5 nm-thick Ni mesh on a graphene layer (“Ni doping”), and 150-nm-thick Ag mesh and 5-nm-thick Au mesh on a graphene layer (“Au doping”). For “ITO” samples, we made patterns using negative PR (AZ 2035) and deposited the 150-nm-thick ITO layers on the patterned p-GaN layers by e-beam evaporation (E-beam). After the PR lift-off, the ITO layers were annealed by rapid thermal annealing at 600 °C for 5 minutes under ambient air. We subsequently completed the metal mesh in all of the TCLs  by metal deposition process and graphene transfer process, as can be seen in Fig. Figure 17/cm3 and 7 × 1017/cm3 for the two samples. We employed lower hole concentration for investigating the different locations of the graphene and higher hole concentration for verifying the doping of the graphene in the TLM samples.We grew two TLM samples with a structure of 1-µm-thick pure p-GaN layers/2-µm-thick un-doped GaN layers on sapphire substrates. We measured the hole concentrations of the pure p-GaN layers using Hall measurements (model: HL 5500 supplied by BIO-RAD) and obtained values of 3.7 × 102 on the pure GaN layers to make mesa masks using PECVD and made mesa patterns on the SiO2 layers using photolithography with AZ 4330. We formed the mesa structures by ICP etching and stripping out the PR. We removed the rest of the SiO2 layers with HF solution. Using the mesa structures, we then fabricated TLM samples with 100-μm width and 200-μm length with a range of gaps from 5-μm to 40-μm with a 5-μm interval. We prepared ITO and Ag samples for the TLM samples (“ITO” and “Ag”) using the same methods described in method section 1.We deposited 300-nm-thick SiOIn order to verify how the contact properties differed depending on the graphene locations, we made two different types of TLM samples: one with a graphene layer on 150-nm-thick Ag (“GR on Ag”) and one with 150-nm-thick Ag on a graphene layer (“Ag on GR”). For “GR on Ag”, we deposited a 150-nm-thick Ag layer on the pure p-GaN layer after creating the TLM patterns and then transferred a graphene layer onto the Ag layer. For “Ag on GR”, we transferred a graphene layer onto the mesa structures of the pure p-GaN layers and deposited a 150-nm-thick Ag layer on the graphene layer. We completed the TLM samples by isolating the graphene layers using reactive ion etching (RIE) with TLM PR mask patterns and removing the rest of the PR.Using the same methods introduced in Sec. 2. 2-1, we made mesa structures on the pure p-GaN layers. To verify how doping the graphene affected the contact properties through thin metal interlayers, we prepared three types of thin metal interlayer on the graphene layer: 150-nm-thick Ag/5-nm thick Ag/graphene (“Ag doping”), 150-nm-thick Ag/5-nm-thick Ni/graphene (“Ni doping”), and 150-nm-thick Ag/5-nm-thick Au/graphene (“Au doping”).For all TLM samples, we transferred the graphene layers onto the mesa structures of the pure p-GaN layers and deposited the thin metal layers by E-beam. Furthermore, 150-nm-thick Ag layers were equally deposited on the thin metal layers. The graphene layers were separated by RIE etching with TLM PR mask patterns, and the rest of the PR was removed.To investigate the doping effect of graphene based on Raman shift, we prepared three types of graphene layers located on different thin metal layers, graphene on 5-nm-thick Ag (“Ag doping”), graphene on 5-nm-thick Ni (“Ni doping”), and graphene on 5-nm-thick Au (“Au doping”), as shown in Fig. Supplementary Information


An optical device configuration allowing efficient electrical tuning of near total optical absorption in monolayer graphene is reported. This is achieved by combining a two-dimensional gold coated diffraction grating with a transparent spacer and a suspended graphene layer to form a doubly resonant plasmonic structure. Electrical tuneability is achieved with the inclusion of an ionic gel layer which plays the role of the gate dielectric. The underlying grating comprises a 2-dimensional array of inverted pyramids with a triple layer coating consisting of a reflective gold layer and two transparent dielectric spacers, also forming a vertical micro-cavity known as a Salisbury screen. Resonant coupling of plasmons between the gold grating and graphene result in strong enhancement of plasmon excitations in the atomic monolayer. Plasmon excitations can be dynamically switched off by lowering the chemical potential of graphene. Very high absorption values for an atomic monolayer and large tuning range, extremely large electrostatically induced changes in absorption over very small shifts in chemical potential are possible thus allowing for very sharp transitions in the optical behavior of the device. Overall this leads to the possibility of making electrically tunable plasmonic switches and optical memory elements by exploiting slow modes. The demonstrated high quantum efficiency for light matter interactions as well as its strong optical nonlinearity, high optical damage threshold, and most importantly the ability to easily tune its optical properties, promotes graphene as a highly promising candidate for photonic devices5. Nevertheless, even though 2.3% of optical absorption is impressive for an atomic monolayer, it is not enough for real world applications which require much higher absorption values for efficient operation. Enhancing light absorption in graphene while maintaining the ability to have strong and dynamic switching or efficient frequency modulation of this absorption is crucial for the commercialization of graphene based active photonic devices.Graphene is an atomic monolayer consisting of carbon atoms tightly packed in a two-dimensional honeycomb lattice. As a promising newly discovered material combining remarkable electronic, photonic and mechanical properties graphene has attracted a lot of attention in the scientific community7, and ribbons8, positioned a small distance from a metallic ground plate. These take advantage of interference effects provided by the spacer geometry helping to dissipate incoming radiation. Additionally, patterned graphene has been used as a light trapping component that can enhance optical absorption in surrounding absorptive mediums9. Attenuation total reflection multilayer structure configurations (prism/graphene/Quartz) have also been used to significantly enhance light absorption in graphene at optical frequencies10. In such devices coherent absorption is achieved by utilizing interference effects in a multilayer structure giving rise to a large electric field intensity in the vicinity of the graphene layer. Hetero-structures devices have also been theoretically suggested to achieve perfect absorption in the THz region by taking advantage of the strong photon localization in an introduced graphene defect layer11. Furthermore, graphene/MgF2 multilayer stack unit cell arrays on Au film plane have been theoretically predicted to achieve a dual-band strong absorption effect12.Excellent progress has recently been made to achieve enhanced light absorption in graphene. Perfect graphene absorbers have been proposed based on graphene disks13 have also been proposed to enhance optical absorption in graphene since they can provide strong field enhancement in the vicinity of the monolayer when carefully optimized. Such devices have effectively been used to enhance interband absorptions in graphene by a factor of 5.5 in the mid-infrared spectral region, also providing a very strong modulation efficiency of 3.3% of absorption change per Volt but provide limited amount of absorption enhancement and wavelength modulation14. It has been theoretically predicted that under grazing angle for s polarized incident light such Salisbury screen/graphene setups can provide nearly total optical absorption, also achievable when using multiple stacked graphene films, but wavelength tuning remains weak15. Absorption due to plasmon excitations in graphene nano-resonators have also been suggested to be enhanced when utilizing Salisbury screen setups, also allowing for absorption tuning by electrostatic gating, but the absorption efficiency remains low (24.5%)16.Devices implementing a Salisbury screen17 or cross-shaped19 metallic resonators positioned above a graphene layer can, at certain resonant conditions, be used as light trapping components for increasing the interaction of light with graphene, thus achieving strong optical absorption. Similarly cross and ribbon shaped metallic resonators combined with double layer graphene wires have been theoretically studied achieving similar results20. Furthermore, metamaterial resonators like split rings21 and interdigitated structures22 have been combined with single or multilayer graphene stacks providing high speed modulation capabilities as well as strong absorption modulation. Theoretical and experimental studies where graphene is used to fill several periodic metallic apertures of subwavelength sizes have also demonstrated high optical absorption24. Finally critical coupling with photonic crystal slab guide mode resonances has been predicted to achieve strong absorption25.Hybrid metal-graphene structures have been studied by several independent groups demonstrating strong optical absorption enhancement in graphene. Theoretical studies have shown that ribbonNevertheless, achieving strong optical absorption in monolayer graphene in combination with the ability to dynamically tune the wavelength of absorption over a large spectral range and especially away from mid infrared and THz frequencies remains a difficult but highly anticipated task. In this work, a device capable of overcoming these limitations by providing efficient electrical modulation of nearly total optical absorption over a large range of wavelengths while allowing operation even at Mid-infrared and almost visible light frequencies is presented.26. In contrast to previous works, a continuous graphene layer, instead of micro/nano patterned graphene, allows excitation of propagating modes instead of localized modes. Such propagating modes typically have higher field confinement in the surface normal27.Figure 28). Using an ionic gel as the top dielectric overcoat material in a gate setup29 enables strong modulation of the chemical potential at low voltage operation (from 0 eV up to 0.8 eV within 3 V of applied voltage)30 thus providing a wide wavelength tuning range for plasmon excitations in the graphene layer. The inclusion of the spacer and ion gel layers also results in the formation of a weak 1-dimensional micro-cavity transverse to the grating surface that is also known as a Salisbury screen13.Optical absorption by the monolayer is further enhanced by positioning Graphene in close vicinity to a gold layer that also allows propagation of plasmons. The structure consists of a conformal two-layer metal-dielectric coating deposited over the underlying diffraction grating. The role of the dielectric spacer is to separate the graphene layer from the gold back reflector but can also be used as a back-gate in order to modulate the chemical potential of graphene. An ionic gel layer is further placed on top of graphene in order to provide a highly efficient gating method for achieving strong electrostatic doping of the atomic monolayer , associated with the periodic gold coated pyramid structure. Similarly, surface plasmon modes are excited on the suspended graphene layer. Due to the geometry of the structure and the excitation of the gold plasmon mode, at the edges of the pyramid groove a strong near field is produced that is extended upwards and strongly interacts with the plasmon mode in graphene Fig. . This reMost importantly the resonance frequency can be adjusted by changing the chemical potential of the graphene layer. Using realistic chemical potentials for the graphene layer it is demonstrated that optical absorption can be tuned from mid infrared frequencies to near-infrared wavelengths . The rate of wavelength tuning for plasmon absorption was found to be 100 nm/0.05 eV of change in graphene chemical potential (estimated 1 μm/0.5 V at the near infrared region).Rigorous coupled-wave-analysis (RCWA) simulations allowed step by step analysis of the multi-stage plasmon coupling mechanisms, confirming strong enhancement of graphene plasmon modes over a spectral range of a few hundred nm.The wavelength of the gold plasmon peak as well as of the diffraction and Salisbury screen features can be determined by geometric parameters of the pyramid structures and of the vertical micro-cavity such as: spacer and ionic gel layer thickness and index of refraction. Optimization of these parameters yielded 1650% increase in absorption of incident light by the graphene mono-layer, compared to a non-resonant device (not implementing the proposed graphene-gold hybrid structure), while maintaining a large tuning range  and excellent electrical modulation efficiency.Since the graphene absorption peak is spectrally very narrow and electrical gate control voltage low, our configuration potentially allows rapid dynamic switching between high and low absorption states by electrical modulation, providing a new device for fast plasmon switching in planar light circuits (PLC). Plasmon frequency can be also adjusted electrically by the gate arrangement, making it suitable for tunable chemical sensors and photonic circuits.2 bonds5. The band structure of graphene is formed by overlapping pz orbitals of neighboring carbon atoms. Bonding π states form the valence band, and the antibonding π* states form the conduction band giving a linear dispertion6 for energies bellow ~1.5 eV. Conduction and valance bands touch at six points known as the Dirac points and can be very accurately approximated as two symmetric cones for low energies. For undoped graphene the Fermi level lies exactly on the Dirac point  and so there is a very small number of free electrons. This in combination with the lack of a band gap allows for direct dynamic tuning of the carrier concentration and position of the chemical potential through electrostatic gating31. Electrostatically tuning the carrier concentration in graphene allows for direct control of complex permittivity of the monolayer. Since plasmon dispersion is directly related to the relative permittivity of the conductor in comparison to that of the surrounding dielectric material, direct electrostatic control of Plasmons is possible.Graphene is an atomic monolayer consisting of carbon atoms arranged in a honeycomb lattice formed by sp26. Phase matching conditions (and thus the excitation of plasmons) can be estimated by assuming a 1D grating structure.ksp the wave vector of plasmons in Graphene, grating order vo is an integer , α the pitch of the grating structure, and kx = ksinθ the in plane wave vector of impinging photons.Under normal conditions incident photons cannot couple to and excite plasmons due to the wave vector of plasmons being higher than that of photons for any given frequency. Mode-matching techniques can be used to provide phase matching between incident photons and plasmons. In this case a complex underlying 2D inverted pyramid diffraction grating is used for coupling plasmons on the graphene layer  spacer layer, with the absorbing graphene mono-layer placed on top. This setup allows for transfer energy from the excited gold plasmon mode to the graphene plasmon mode, thus increasing the efficiency for coupling light in the atomic monolayer. Once again an ionic gel layer serves the role of the transparent gate dielectric for controlling the chemical potential of graphene. The inclusion of the spacer and ITO layers also forms a flexible vertical Salisbury screen micro-cavity (more information in the supplementary document).In our device, optical absorption of the graphene layer can be enhanced by coating the grating with a conformal reflective gold layer and a transparent dielectric   Fig. .Figure 3Detailed discussion of Fig. is given later).As seen in Fig. Figure From the contour plot in Fig. It is important to mention that in this case the Salisbury screen cavity mode has been tuned to overlap with the gold plasmon mode. Due to the inclusion of the pyramid structures and the limited amount of flat area on the surface of the device the vertical Salisbury screen cavity only contributes to a few percent of the total optical absorption from the device and thus it is difficult to observe from this graph. The absorption of the Salisbury screen has been identified as an isolated absorption mechanism in the device and does not contribute in energy transfer towards the graphene layer. More information about the absorption due to the Salisbury screen cavity is provided later on.Absorption attributed solely to the graphene layer can be de-convolved from the data by subtracting equivalent results for the multilayer structure without the graphene layer. This is plotted in Fig. The enhancement wavelength range can be tuned by changing the thickness or refractive index of either of the transparent layers (spacer/ionic gel). An example of this can be seen in Fig. It is important to mention that the modulation of plasmon frequency is very strong, demonstrating a wavelength shift of about 100 nm per 0.05 eV of chemical potential change in graphene (estimated 1 μm/0.5 V at the near infrared region - more details can be found in the supplementary information) allowing for extremely low voltage tuning of absorption over a broad spectral range. Combining strong absorption and effective dynamic tuning over a large spectral range is ideal for fabrication of highly efficient modulators. Additionally the plasmon excitation can be effectively shut down by lowering the chemical potential (voltage) in graphene thus providing capability for dynamic electro-optical switching.We now de-convolve the physical processes underway on a step by step basis. Figure 34.The set of sharp (but very feint) curved lines (labeled with red dashed lines) correspond to dispersion modes associated with the underlying dielectric pyramid grating structure. These relate to phase matching conditions coupling vertically incident light from free space to lateral propagating modes in the dielectric structure via a scattering/diffraction mechanisms1 the thickness of the spacer and ns1 the value of the spacer refractive index, ds2 the thickness of the ion gel and ns2 the refractive index of the ionic gel, and m is an integer cavity mode number.The dark fringes result from destructive interference in the dielectric bi-layer stack, whereas the broad light fringes are associated with weakly resonant modes of the vertical micro-cavity resulting from constructive interference . The conditions for destructive interference in this vertical cavity are:s2 as black dashed lines revealing perfect agreement with the diagonal bright fringes seen in the RCWA simulation. Hence Fig. m, hence the fringes become periodic with cavity width.Figure 35.With the inclusion of the gold back reflector the diffraction lines previously associated with the dielectric grating  correspond to resonant coupling conditions (perfect phase matching) between plasmons supported by the underlying periodic gold pyramid structure, and surface plasmons supported by the graphene film (green dashed circle marker). Energy transfer to the graphene layer, and thus higher absorption in the monolayer, was found to occur only when the graphene plasmon peak overlaps or is in close vicinity to the gold plasmon peaks.On the contrary, when the graphene peak overlaps with diffraction features (marked with red crosses), or features originating from the Salisbury screen vertical cavity (marked by black dashed lines), the amount of absorption attributed to the graphene monolayer is not increased. When features overlap with the Salisbury screen peak the overall absorption from the device increases (as marked by the white dashed circle marker). It is important to mention though that this increase is a simple additive effect and there is not transfer of energy between the individual effects and the vertical micro-cavity mode  correspond to zero and 1st order diffraction34, whereas curved lines (black dashed overlay lines) correspond to dispersive modes associated with the 2-dimensional grating geometry. Points along these lines correspond to conditions whereby incident light become coupled to the lattice resulting in propagating waves in the lateral direction (in the plane of the lattice). Simple crossing points between bands are observed rather than points of inflection (which would indicate presence of mini stop-bands). This is because the lattice pitch of the underlying structure is relatively large hence photonic band gaps are not observed, just continuous dispersive and diffractive modes. One point to note, no features associated specifically with the Salisbury screen micro-cavity are observed in this situation because without the gold back reflector light mainly passes through the substrate  which can be wavelength tuned by adjusting chemical potential. The plasmon bands are much broader than the dielectric dispersion lines (black and white dashed overlay lines). Points of intersection between the graphene and gold plasmon bands correspond to perfect phase matching conditions (x-axis of this diagram relates to in-plane phase) allowing efficient transfer of energy between the structures.Figure Figure Figure Figure 0–180 the fundamental (lowest order) plasmon band (dashed green line) does not change wavelength with angle of incidence, has zero gradient, and therefore corresponds to a zero group velocity (stationary) plasmon mode which is localised to the top surface of the gold between the pits. This results in strong single step coupling of incident light over a wide range of surface incidence angles, and allows efficient coupling of light directly to the fundamental plasmon mode from a high NA focussing lens.Inspecting Fig. By adjusting the chemical potential, the graphene plasmon resonance  wave, it can be predicted that a very small change in chemical potential would rapidly detune the coupling conditions blocking optical energy transfer to the graphene, hence we propose that this design provides basis for fast electro-optical switching which could be exploited in as an optical modulator, or as an optical memory element.Referring back to Fig. In conclusion efficient electrically tunable near total optical absorption in a device implementing monolayer graphene is reported. The combination of a two-dimensional diffraction grating with a hybrid gold-insulator-graphene multilayer setup forms a doubly resonant plasmonic structure providing strong absorption enhancement in the graphene film. This enhancement can even provide a 1650% percent of absorption increase in the graphene layer when compared with a device not implementing the proposed hybrid setup. The physical mechanism behind this enhancement is a combination of increased diffraction efficiency for the grating structure, and resonant coupling between plasmons generated in the gold layer to the ones generated in Graphene. The frequency of plasmon excitations in the graphene layer can be controlled electrostatically with the use of an ionic gel layer used in a gate configuration. Tuning is highly efficient allowing for an estimated 1 μm/0.5 V shift of plasmon absorption wavelength and allowing the device to operate at near-infrared frequencies. Furthermore, the absorption peak that is due to graphene plasmons is spectrally narrow and in combination with the efficient electrostatic control can potentially allow for rapid dynamic switching between high and low absorption values. Most importantly, plasmon excitations can be switched off completely by lowering the chemical potential and moving to the inter-band transition region. Finally, a near zero group velocity plasmon mode was found to be excited in the Au layer over a large range of angles thus allowing optimal coupling over a wide range of incidence angles as would for a large NA focusing lens. Crucially, this configuration allows for probing a static non- travelling wave thus potentially providing the basis for fast electro-optical switching to an optical memory element. The device presented in this work has potential in enabling a variety of tuneable nano-photonic devices including sensors, photonic logic gates, optical interconnects, and electro-optical memories.Supplementary information


We review our recent time- and angle-resolved photoemission spectroscopy experiments, which measure the transient electronic structure of optically driven graphene. For pump photon energies in the near infrared ( Dynamical engineering of the electronic structure of solids with light is emerging as an important new area of research, which complements more traditional routes of control using chemistry, pressure, or magnetic fields. Especially interesting are new techniques that can drive materials close to lattice and other resonances at mid-infrared and THz frequencies. Indeed, in these driven solids, new properties emerge away from equilibrium. These result naturally from modulation with a strong, periodic field that creates new effective Hamiltonians with new eigenstates.3C60,2Se3.Resonant excitation of the crystal lattice has, for example, been used to induce superconductivity far above the equilibrium critical temperature in cuprates1u phonon in bilayer graphene at Eph = 200 meV, in addition to carrier heating via free carrier absorption, the electronic band structure is modulated by the coherent oscillation of the carbon atoms. In this case, we observe anomalous carrier dynamics that we attribute to a transient enhancement of the electron-phonon coupling constant  For pump photon energies bigger than twice the chemical potential, electron-hole pairs are generated. Photo-excited electrons are found to rapidly accumulate above the Dirac point at the bottom of the conduction band where they form a short-lived population-inverted state  substrates were hydrogen-etched and annealed in an Argon atmosphere, resulting in carbon coverages of one and two monolayers, respectively. These carbon layers were then decoupled from the underlying substrate by hydrogen intercalation, giving quasi-freestanding monolayer and bilayer graphene.μm wavelength are typically used to excite the sample. At a variable delay from this excitation, the sample is illuminated with a second extreme ultra-violet (XUV) probe pulse that ejects photoelectrons from the sample. These photoelectrons are measured with a hemispherical analyzer, yielding the photocurrent as a function of kinetic energy, Ekin, and emission angle, θ. From the kinetic energy, the binding energy, EB, of the electrons in the solid can be obtained, and the measured emission angle can be converted into in-plane momentum, Tr-ARPES measurements have been performed at the Artemis user facility at Harwell, United Kingdom. Femtosecond laser pulses tunable between 800 nm and 15 μm) with subsequent difference frequency generation (4 to 15 μm). About 1 mJ of laser energy is used for high-order harmonics generation in an Argon gas jet, generating XUV photons in the energy range between 10 and 40 eV. A single-grating time-preserving monochromator is used to select a single harmonic at μm.The setup is based on a 30 fs/780 nm/1 kHz Titanium:Sapphire (Ti:Sa) laser. Pump pulses are generated with a commercial optical parametric amplifier  is too bad to allow access to the primary thermalization events.The measurements displayed in Fig. The ultrafast <30 fs dynamics after photo-excitation are predicted to be dominated by different Auger processes as sketched in Figs. CB, as well as their average kinetic energy, ECB/NCB. In the case of Auger recombination, NCB decreases while ECB/NCB increases and vice versa for impact ionization. These numbers are directly accessible in a tr-ARPES experiment, as the photocurrent is proportional to the number of electrons at any given energy.These two processes can be easily identified by measuring the number of carriers inside the conduction band, Nσ = 9 fs (FWHM = 21 fs) using a pump photon energy of CB as well as ECB/NCB as a function of pump-probe delay. We find that, during the first ∼25 fs after photo-excitation, NCB keeps increasing while ECB/NCB already decreases, clearly indicating impact ionization.In order to resolve these primary scattering events, we repeated the experiment with a temporal resolution of For experiments in which the pump photon energy was below the threshold for interband excitation, metallic carriers in our hole-doped samples absorb energy from the pump by phonon-assisted intraband absorption Fig. . This ex1u phonon in bilayer graphene at Eph = 200 meV. (Monolayer graphene does not possess any infrared-active modes and cannot be used for the present purpose.) In this case, in addition to carrier heating via free carrier absorption  the observation of a transient population inversion, (2) pre-thermal scattering events dominated by impact ionization, and (3) anomalous dynamics due to resonant excitation of the crystal lattice.a priori clear whether such a state might exist in graphene. Ryzhii et al.et al.(1) and (2) are related to the conical band structure of graphene and the vanishing density of states at the Dirac point. While the occurrence of a population-inverted state after strong optical excitation is commonly observed in conventional semiconductors with a sizable band gap, it is not In our tr-ARPES experiments performed in different excitation regimes and with different temporal resolutions, we are able to observe the complete dynamics during the buildup and decay of the population inversion.1u phonon is resonantly excited can be understood in terms of a transiently enhanced electron-phonon coupling constant.1u mode.The observed anomalous carrier dynamics that occur when the EThis can also be understood from intuitive arguments without the need to invoke theoretical models. An increase in electronic temperature increases the phase space for optical phonon emission and is expected to result in faster cooling of the electronic system.1u mode in bilayer graphene leads to faster relaxation at lower peak electronic temperatures that we attribute to a transient enhancement of the electron-phonon coupling constant.We have reviewed the electronic-structural dynamics of optically excited graphene at different pump wavelengths throughout the near- and mid-infrared. These dynamics were tracked with time- and angle-resolved photoemission spectroscopy. We find that free-carrier absorption efficiently heats the electrons to high temperatures but maintains a thermal electronic distribution at all times. After interband excitation, carriers rapidly accumulate at the bottom of the conduction band, resulting in a short-lived population-inverted state. Using sub 10 fs pulses, we were able to show that the predominant scattering channel during the first 25 fs after interband excitation is impact ionization. Finally, resonant excitation of the EWhile the observed short-lived population inversion may in principle be used to amplify TeraHertz pulses, the presence of impact ionization most likely cannot be exploited for efficient light harvesting as the absence of a band gap in graphene makes electron-hole separation difficult.3C60.The observed transient enhancement of the electron-phonon coupling strength presents a significant step towards dynamical band structure engineering with important consequences for the occurrence and the understanding of light-induced superconductivity in graphene and other carbon compounds, such as K


To develop effective question understanding tools, question corpora semantically annotated for relevant question elements are needed. In this paper, we present a two-part consumer health question corpus annotated with several semantic categories: named entities, question triggers/types, question frames, and question topic. The first part (CHQA-email) consists of relatively long email requests received by the U.S. National Library of Medicine (NLM) customer service, while the second part (CHQA-web) consists of shorter questions posed to MedlinePlus search engine as queries. Each question has been annotated by two annotators. The annotation methodology is largely the same between the two parts of the corpus; however, we also explain and justify the differences between them. Additionally, we provide information about corpus characteristics, inter-annotator agreement, and our attempts to measure annotation confidence in the absence of adjudication of annotations.Consumers increasingly use online resources for their health information needs. While current search engines can address these needs to some extent, they generally do not take into account that most health information needs are complex and can only fully be expressed in natural language. Consumer health question answering (QA) systems aim to fill this gap. A major challenge in developing consumer health QA systems is extracting relevant semantic content from the natural language questions . Problems are the most frequent named entities, while treatment and general information questions are the most common question types. Inter-annotator agreement was generally modest: question types and topics yielded highest agreement, while the agreement for more complex frame annotations was lower. Agreement in CHQA-web was consistently higher than that in CHQA-email. Pairwise inter-annotator agreement proved most useful in estimating annotation confidence.To our knowledge, our corpus is the first focusing on annotation of uncurated consumer health questions. It is currently used to develop machine learning-based methods for question understanding. We make the corpus publicly available to stimulate further research on consumer health QA. Consumers increasingly rely on the Internet for their health information needs . A recenquestion understanding for information retrieval systems. Question understanding can broadly be defined as the task of extracting relevant elements or generating a formal representation of a natural language question, which can then be used to formulate a query to a search engine and retrieve relevant answers  ,they found repot was PROGRESSIVE DISEASE,ACTIVE LUNG TBINTHE RIGHT B2 AND B4 SEGMENS,THE EXUDATIVE LESIONS IS INCREASING WITH SMALL CAVITY.so what we have to do for her is this contages,who is the pople staying with her need to do test ?pls guide me thank u my contact [CONTACT]Most research in biomedical QA has focused on clinical questions asked by healthcare professionals , which acontagious is misspelled, making both sub-questions very difficult to parse automatically. Based on such characteristics, it has been suggested that consumer health QA systems should be designed with considerations different from those underlying QA systems targeting healthcare professionals  in the Raf gene are associated with cancer?). More recently, BioASQ challenge also focused on biomedical QA and presented a corpus of 311 questions, categorized by the type of answers they require: yes/no, factoid, list, and summary ,they found repot was PROGRESSIVE DISEASE,ACTIVE LUNG TBINTHE RIGHT B2 AND B4 SEGMENS,THE EXUDATIVE LESIONS IS INCREASING WITH SMALL CAVITY.so what we have to do for her is this contages,who is the pople staying with her need to do test ?pls guide me thank u my contact [CONTACT]We briefly discussed several approaches to biomedical question representation above: empirical, domain-based, and formal. Our approach to representing consumer health questions can be viewed as a hybrid method. Like empirical approaches, we begin with existing questions and categorize them into a limited number of generic question templates, and, like formal approaches, we create structured representations of the information in these generic questions that we call pecified . A questdo in what we have to do for her. Note that while a THEME argument is required for each frame, we only specify other arguments if they are likely to be relevant for answer retrieval. For example, in the first frame, one could specify woman as an argument with PATIENT role ; however, since gender is unlikely to be significant in answering this particular question, its explicit representation is not required. In this sense, our representation is driven more by pragmatic concerns for question answering than by completeness.The first frame indicates that the information request contains a question about the treatment of active lung tuberculosis, triggered by the verb THEME of a sub-question) and the question types in an information request are known, authoritative answers can be found in 60% of the cases [ACTIVE LUNG TB is marked as the question topic. The annotation of this question is illustrated in Fig Our prior research suggested that when the question topicOur corpus consists of two parts: a set of 1740 questions harvested from consumer health emails received by the NLM customer service between 2013 and 2015 (CHQA-email) and another set of 874 questions harvested from MedlinePlus search query logs (CHQA-web).CHQA-email is an extension of an earlier corpus (CHQ-NER), reported in , and supAs part of the present study, we extended CHQ-NER with 195 additional questions. Furthermore, 2 duplicate questions and 1 non-question request from the original corpus were dropped, bringing the total number of questions to 1740. One hundred fourty six of the new questions come from the set of 300 questions that were considered in a study that investigated whether authoritative answers to consumer health questions can be found in MedlinePlus and other online resources . In thatTHEME, KEYWORD, and EXCLUDE_KEYWORD), and prepared annotation guidelines. We did not calculate inter-annotator agreement for these practice questions and only make available the final, adjudicated annotations.The rest of annotation of this part of the corpus proceeded as follows: first, all annotators annotated 20 questions for the elements under consideration . As the question type categories, we used those that were used in annotating GARD questions . As thisTHEME argument, except when the question type is one that allows multiple THEME arguments, such as COMPARISON.Next, we sliced the rest of corpus (1720 questions) such that each annotator was allocated roughly the same number of questions, and each shared with all the other annotators approximately the same number of questions. Seven annotators were involved, although the additional annotator (Ann7) annotated significantly fewer than the others due to scheduling conflicts. The questions initially allocated to her but could not be annotated were redistributed among four annotators in such a way to ensure that each question was still double-annotated and the questions were fairly distributed. The resulting number of questions annotated by each annotator is given in Table 1 score when one set of annotations is considered the gold standard [We calculated inter-annotator agreement for question triggers/types, frames with and without optional elements, and question topic. Inter-annotator agreement is calculated as the micro-average Fstandard .Considering the length of questions, the average number of annotations per question, and the number of annotators involved, adjudicating this set is a daunting taskwh-words to identify these questions in the query logs: how, what, when, where, which, who, why. We also included queries ending with question marks and those that start with an auxiliary . For each category, we selected 70 most frequent questions as well as 70 random questions. After removing duplicates and near-duplicates using string matching heuristics, we were left with 1180 questions. In contrast to CHQA-email, we did not assess whether these were all legitimate questions or were answerable; instead, we used two labels, NOT_ANSWERABLE and NOT_QUESTION, during the annotation process to mark such questions. Recognizing that our heuristics for removing duplicates may not be adequate, we used another label, DUPLICATE, to indicate such cases.In contrast to CHQA-email, the second part of the corpus, CHQA-web, has been fully annotated from scratch in this study. MedlinePlus provides search capabilities that are geared towards traditional keyword searches; however, a significant number of searches are posed as natural language questions. Such searches often fail. To investigate whether we can address such questions automatically and to create a counterpoint to customer service information requests, we harvested a set of short questions from MedlinePlus search query logs. These questions were asked in the period from May 2015 to April 2016. We looked for In a practice step, the same six annotators annotated and adjudicated 100 of these questions using the guidelines for CHQA-email. During this process, the guidelines were updated: some question types were consolidated, a couple of new types were added, and the types of semantic roles were expanded.1 score. Fourty eight questions were adjudicated as NOT_QUESTION, 203 were deemed NOT_ANSWERABLE and 55 were found to be DUPLICATE, bringing the total number of annotated questions in this part down to 874.Next, four of the annotators double-annotated the rest of the questions . Instead of pairing all annotators, we only paired those with clinical expertise (n=2) with those without (n=2), leading to four annotator combinations. In the end, each annotator labeled 540 questions. Since these questions were relatively short and there were fewer annotator pairs, we chose to adjudicate these annotations. Each pair adjudicated the questions that they both annotated. As in CHQA-email, we calculated inter-annotator agreement using micro-average Fbrat annotation tool [brat. Question topic was specified as an attribute of the relevant named entity.Seven annotators participated in various stages of the annotation, depending on their availability. All annotators had experience in working with biomedical text. Two annotators  have clinical expertise, two are trained as medical librarians , and four have experience in natural language processing . We used ion tool  for all In the rest of this subsection, we provide details about annotation of these semantic layers and highlight the differences in their annotation between the two parts of the corpus.Named entity annotation in CHQA-email essentially follows the principles of CHQ-NER .ANATOMY, CELLULAR_ENTITY, DIAGNOSTIC_PROCEDURE, DRUG_SUPPLEMENT, FOOD, GENE_PROTEIN, GEOGRAPHIC_LOCATION, LIFESTYLE, MEASUREMENT, ORGANIZATION, PERSON_POPULATION, PROBLEM, PROCEDURE_DEVICE, PROFESSION, SUBSTANCE, and OTHER). Nested entities and multi-part, non-contiguous annotations were allowed. Annotation was performed both manually and with assistance from several named entity recognizers [In CHQ-NER, we annotated 16 broad categories of named entities , we added a new named entity type ORGANISM_FUNCTION. We use the UMLS [OTHER in CHQ-NER and identified those that can be considered organism functions. We then automatically relabeled these mentions, originally labeled OTHER, as ORGANISM_FUNCTION in CHQA-email.For the present study, recognizing that many of the entities with the UMLS  definitiORGANISM_FUNCTION and RESEARCH_CUE. The latter, though not common, is a category that was devised to indicate that the consumer is interested in information that goes beyond typical consumer health information.For CHQA-web, we considered 18 categories: 16 categories above in addition to RESEARCH_CUE mentions include new and latest information. In contrast to CHQA-email, we did not consider nested entities for CHQA-web, as the annotators generally found the annotation of nested entities challenging, although such entities can provide a more precise evaluation for named entity recognizers, as illustrated in [Example rated in .The full list of named entity categories used in annotation, with brief definitions, examples, and relevant UMLS semantic types, is provided in Table ACTIVE LUNG TB is the question topic, while there are two topics in the following question in bold: (4)I’d like to learn more aboutmegalocytic interstitial nephritiswithmalakoplakia.Question topic is the central theme  of a question. Each legitimate question is expected to have at least one topic, but multiple topic elements can be annotated, especially in questions about relationships between two diseases or drugs. In the earlier example, anosmia and loss of smell are both legitimate topics. (5)ANOSMIA.This site tells me that ongoing research may solve my problem some day. Please, could I be informed if such result does happen? I live alone, aged 78, and this completeloss of smellhas been my great problem for quite a while.All mentions of the topic term, including synonyms and misspellings, were expected to be annotated. For example, in the following question, synonyms ropinirole HCl as well as repinirole HCl 0.25 mg and ropinirole HCl 0.5 mg were annotated as named entities. With this instruction, annotators were expected to label mentions of ropinirole HCl as the topic, since the dosage/form of this drug is not relevant to the primary question. (6)Ropinirole HCl 0.25 and ropinirole HCl 0.5 mg. Please tell me if these meds are gluten-free.When considering nested entities for question topics, annotators were instructed to label the term as topic at the nesting level that is appropriate for the question. For example, in the following question, both mentions of THEME argument of the question frame.These principles applied to question topic annotation in CHQA-email. For short questions in CHQA-web, based on the practice annotation, we chose not to explicitly annotate question topic, since this almost always corresponded to the General question types: These categories can apply to any type of entity .Problem question types: These categories apply to diseases/conditions, and other problems . A subset of these categories also apply to entities of procedure and medical device types, since problems can arise from their usage .Intervention question types: These categories apply to drugs, supplements, as well as procedures and medical devices .We based our practice annotation of CHQA-email on the 14 question type categories proposed for disease questions in . DiscussWe provide some further details in Table SUSCEPTIBILITY as defined in Roberts et al. [FREQUENCY, INHERITANCE, and SUSCEPTIBILITY. Considering that answers to such questions are likely to be found in the same passages, for annotation convenience and expediency, we merged these types back into SUSCEPTIBILITY in CHQA-web. The consolidated question types are the following: {complication, long_term_effect, overdose, side_effect} →complication{person_organization, support_group} →person_organization{frequency, inheritance, susceptibility} →susceptibility{dosage, storage_disposal, tapering, usage} →usageFor question type annotation in CHQA-web, this scheme was modified in several ways, resulting in a somewhat simplified categorization that consists of 26 classes. First, some question types that do not occur frequently were merged into other categories that can be considered their supertypes. For example, in CHQA-email annotation, we split the question type s et al.  into thrEFFECT to indicate underspecified relations between two or more named entities, and changed its name to ASSOCIATION for clarity. Finally, we added two new question types. DIAGNOSE_ME identifies questions asking for diagnosis from a set of symptoms. For example, this question type was annotated for the following question (the trigger is in bold). (7)my waist turn red like i have hives but it’s not and its not achy or itchy,whatdo u thinkit is?We generalized the question type TIME, identifies questions asking for time/duration of an event. An example is given below. (8)who gets vaccines andwhen?While we specifically avoided annotating such questions in CHQA-email, since they were considered beyond the scope of a QA system provided by a medical library, we added such annotations in CHQA-web to allow further experiments. The other question type, take, rather than can take or if I can take. In the second question, since both reduce and remove can trigger the same question type, the question trigger is taken as reduce or remove. (9)I would like to ask if I cantakeglutathione supplement.fibroadenomas.…taking medicine is help toreduce or removethis?…The annotators were instructed to annotate the minimal trigger expression, except when two equally valid triggers are conjoined. In the first question in Example THEME argument, which corresponds to a named entity. The type of the question frame is inherited from the trigger. Most questions types allow a single THEME argument, with the exceptions of EFFECT, which allows multiple THEME arguments, and COMPARISON, which requires multiple THEME arguments. Restrictions on the type of named entities that can fill the THEME role are specified in Table In CHQA-email, question frames were annotated using the 17 named entity categories and 33 question types discussed above. A question frame in this context minimally consists of two elements: a trigger indicating the question type and a INTERACTION still required at least two THEME arguments. Two additional question types, DIAGNOSE_ME and TIME, were allowed multiple themes and a single theme, respectively.In CHQA-web, question frame annotation involved 26 question types and 18 named entity categories. We lifted the type restrictions for theme arguments for this part of the corpus, because the annotators found them too restrictive. However, the cardinality restrictions remained in place. For example, THEME semantic roles. In CHQA-email, two generic semantic roles were defined: KEYWORD, and EXCLUDE_KEYWORD. KEYWORD arguments indicate named entities that are expected to be helpful in finding more specific answers to a question. EXCLUDE_KEYWORD arguments are those that should be used to exclude some potential answers. While KEYWORD can be viewed as a generic semantic role and can correspond to more typical semantic roles, such as AGENT, PATIENT, PURPOSE, EXCLUDE_KEYWORD is unique and is meaningful in a question answering context. Examples in Table Two parts of the corpus also differ with respect to non-EXCLUDE_KEYWORD was not considered. The fine-grained roles used are as follows: agent indicates the entity that performs the action involving the theme. In the clinical domain, inanimate objects, such as drugs, procedures and devices could be agents. For example, in the first example in Table keyword arguments can be annotated as agents.locative indicates the location where the action involving the theme occurs . leg in the question parkinson’s disease best drugs for legs? can be annotated with this role in a treatment frame.purpose indicates the reason/goal for which the action is performed. eczema in the question how many mg. of zinc daily for eczema patients? can be annotated with this role in a usage frame.patient indicates the experiencer of the action. baby in the question what if a mother quits intravenous drug use at five months what side effect will baby have? (a complication frame).restrictor indicates the context in which the action occurs or a restriction on the type of information being sought. doctor in the question what kind of doctor would i see for calcification? (a person_organization frame).temporal indicates the time at which the action occurs or the duration of the action. 4 years ago in the question is zoster vaccine effective if one had shingles 4 years ago? (an indication frame)research indicates the role of research_cue entities. literature in the question review of literature of cancer stress (an information frame).In CHQA-web, we used fine-grained roles, more in line with typical roles used in semantic role labeling. On the other hand, PATIENT role is likely to be filled with an entity of PERSON_POPULATION type, and the RESEARCH role should be filled with a RESEARCH_CUE entity.A question frame can contain multiple arguments with such optional roles. We did not stipulate any semantic restrictions on the named entities that can fill these roles, even though for example, the RESEARCH to indicate that the person asking the question is interested in information that goes beyond consumer health information. In CHQA-web, as noted above, we specifically annotated terms indicating such information need as named entities (RESEARCH_CUE), in order to allow training of machine learning models that automatically learn such phrases.In CHQA-email, we associated a frame with the attribute When there are multiple mentions of a term that fills a specific role, the annotators were instructed to use the mention that is syntactically linked to the question type trigger, if possible. Otherwise, they select the closest mention that precedes the trigger. Similarly, if there are multiple potential triggers, annotators were expected to select the trigger that is syntactically related to the theme or is the trigger closest to it.(10)questions about the effect of L-Leucine.Hello,do you have any introduction about L-Leucine on the effect of treat cancer?If so,could you kindly tell me details?The same question is sometimes asked multiple times in the same information request, especially in long requests. Annotators were expected to annotate all instances of such questions. We also found that some questions can be represented from multiple perspectives. Consider the example below: treatment frame where cancer fills the theme role and l-leucine is an agent A indication frame where l-leucine is the theme and cancer is a purpose (corresponding to is l-leucine indicated for cancer?)An This can be formalized in one of two ways: l-leucine is the more salient entity  and therefore, the frame in which this entity appears as the THEME element  would be preferred. If both entities are equally likely as question topic, we asked the annotators to prefer specific question types over others .Annotators were instructed to annotate a single frame in such cases and to take the question topic into consideration in making a decision. In this question, It became clear early on that adjudicating annotations in CHQA-email would be significantly difficult, due to the number and length of questions, and the size of annotator pool (21 annotator pairs). This is in contrast to CHQA-web with the smaller number of questions, which are shorter on average, and annotators (4 pairs). Thus, we investigated whether we can estimate confidences for the annotations.For this purpose, we used two annotation confidence modeling methods: MACE (Multi-Annotator Competence Estimation)  and anotWe calculate the mean average confidence score for each semantic class .If the item under consideration is named entity or question trigger, it is always considered consistent, as they are atomic and independent.If it is a question topic, it is considered consistent as long as the underlying named entity has already been added.If it is a question frame, it is consistent as long as its trigger and all its arguments have already been added.If the confidence score associated with an individual item is higher than the mean average for its class, it is added to the new set of annotations as long as it is consistent with the annotations already added. Consistency is defined as follows: We considered annotations of named entities, topic, triggers, and frames for modeling. To simplify matters, we treated each annotation as an independent data instance for MACE and P&C. Once we modeled the annotation confidences using these methods, we used the confidence scores generated to create a new set of annotations. When incorporating these scores, we assumed that if two annotators agreed on a specific annotation, that annotation was correct. For the rest of the annotations on which two annotators did not agree, we followed the following procedure: We compared these two methods to two other simple methods: the first (AGREE) only considers the subset of annotations that annotators agreed upon for the new set. The second (IAA_RANK) uses inter-annotator agreement results. For this method, we rank annotators by their inter-annotator agreement with others. Then, for a given question, we prefer the annotations of the annotator ranked higher among the two who annotated the question. Note that none of the methods as used here can distinguish cases where both annotators agree but are both incorrect.To evaluate the performance of these methods, we ran experiments on CHQA-web, since an adjudicated set of annotations was available. Based on the results we obtained, we generated several sets of annotations for CHQA-email incorporating these methods. We make these sets publicly available in addition to raw annotations by each annotator.In this section, we present and discuss corpus statistics for each part of the corpus, provide an in-depth analysis of the semantic categories annotated, detail inter-annotator agreement and annotation confidence estimation results.Table Statistics regarding named entities in the corpus are provided in Table DRUG_SUPPLEMENT and DIAGNOSTIC_PROCEDURE terms appear more frequently in CHQA-web, while categories like PERSON_POPULATION, ORGANIZATION, PROFESSION, which often provide background information, appear less frequently than in CHQA-email. The distribution also confirms findings from earlier studies that consumer questions focus mostly on diseases and symptoms .As expected, the distribution of named entity categories and the average number of annotations per question in CHQA-email are similar to those reported for CHQ-NER, which CHQA-email supersedes. For the short questions in CHQA-web, the average number of entities per question is much lower (1.6). PROBLEM category had the highest number of additions (141), and ORGANIZATION had the highest number of deletions (12), though additions/deletions often amounted to a very small percentage of the entities in a given category. Additions included problems like pain in period and drug classes like allopathic medicines.A small number of named entity annotations were added to CHQA-email (Unadjudicated) or deleted by annotators, increasing the number of annotations in this set from 33,494 to 33,747. RESEARCH_CUE category was only annotated in CHQA-web. In CHQA-email, RESEARCH attribute assigned to question frames indicates the same information without explicitly marking the mention. The total number of RESEARCH attribute annotations are 2 and 109 in CHQA-email (Practice) and CHQA-email (Unadjudicated), corresponding to 0.8% and 0.3% of all the entities, respectively, indicating a distribution similar to that of RESEARCH_CUE in CHQA-web.TREATMENT triggers are most common in CHQA-email by a significant margin. In CHQA-web, INFORMATION triggers are most common, though its margin over the next most frequent type, CAUSE, is not as large. There are differences in the distribution of question types between the two parts: most significant are PERSON_ORGANIZATION and PROGNOSIS question types, which appear much more frequently in CHQA-email, and CAUSE, LOCATION, ACTION, and INDICATION question types, which occur more frequently in CHQA-web. Some of these differences can be explained by the question selection methodology. For example, in constructing CHQA-web, the same number of questions were sampled using different wh-words. where questions often contain LOCATION questions, which occurs in large quantity in CHQA-web, but is very rare in CHQA-email. The comparison of frequency of question trigger types in CHQA-email and CHQA-web is illustrated in Fig. Statistics regarding question triggers/types in the corpus are provided in Table TREATMENT, COST, and CAUSE question types have the least uniform distribution , while in CHQA-web, these question types are LOCATION and CAUSE. There is a slight preference for verbal triggers in CHQA-web as compared with CHQA-email. Light verbs, such as get, help, have, and do are the most heterogeneously annotated triggers, indicating 13, 11, 11, and 11 different question types, respectively, in CHQA-email. In CHQA-web, these triggers are have, why, when, and take, indicating a mix of light verbs and wh-words, to some extent reflecting the question selection process for this part of the corpus.We analyzed the distribution of trigger expressions for each question type. A subset of results is provided in Table OTHER_QUESTION and DRUG_QUESTION categories in CHQA-email. After the annotations were completed, we manually categorized the small number of proposed types the annotators had specified. These types are provided in Table DIAGNOSE_ME (which we adopted later for CHQA-web) were among the most frequent. In general, the annotators were not consistent in proposing new question types; therefore, these results should be interpreted with caution. Also note that the annotation guidelines sometimes indicate a particular question type for a specific kind of information request annotated with one of these two types ; however, these were missed by some annotators, indicating potential annotation errors.We asked annotators to indicate their proposed question types when annotating SIDE_EFFECT trigger was annotated with highest number of frames, indicating that consumers were often interested in side effects of multiple drugs in the same question. The comparison of frequency of question frame categories in CHQA-email and CHQA-web is illustrated in Fig Statistics regarding question frame types in the corpus are provided in Table THEME role was approximately 68% in CHQA-email and 46% in CHQA-web. This difference is to be expected, as longer CHQA-email questions often provide more context that may be relevant for answering questions. The most frequently used semantic role in CHQA-web was the unspecific RESTRICTOR. Considering that this role is largely similar to KEYWORD and not very informative, fine-grained annotation of semantic roles  may not provide much advantage over simply using coarse-grained KEYWORD when it comes to question understanding. EXCLUDE_KEYWORD role was rarely used in CHQA-email, suggesting that its utility for answering consumer health questions may be limited, and was later dropped for CHQA-web.Table We only annotated question topic in CHQA-email, as this element almost always corresponds to the theme element of the question in the short CHQA-web questions. On average, 1.8 question topics were annotated per question. The maximum number of topic mentions annotated for a single question was 17.1 score when one set of annotations is taken as the gold standard. Agreement was calculated for named entities, question triggers/types, question topics, and frames. Exact span match criterion was used to compare mentions (named entities and question triggers).We calculated pairwise inter-annotator agreement, using micro-average Ffull frame agreement calculation, we considered the question trigger/type and all semantic roles . For core frame agreement, we considered only the THEME role in addition to question trigger/type. In addition, frame agreement calculation can be based on question trigger mentions or normalized question types indicated by these mentions. Combinations of these options, consequently, led to the following frame agreement types: Full frame w/ trigger, Full frame w/ type, Core frame w/ trigger, and Core frame w/ type. We consider Core frame w/ type as the most relevant for QA, as identifying specific words used to express a question precisely is generally not needed for accurately answering it.We calculated frame agreement in several ways. For We did not consider agreement on practice sets for each part of the corpus. We also did not calculate named entity agreement on CHQA-email (Unadjudicated), since this largely overlaps with CHQ-NER and the inter-annotator agreement for named entities in that set were reported earlier . The resFull frame w/ trigger). Frame annotation in general is challenging as the results indicate, but the agreement is somewhat improved when the agreement only focuses on core frame elements, question type and theme (0.41 with Core frame w/ type). The low agreement for frames is not unexpected, as they can involve many parts, each potentially leading to disagreement. In particular, annotators had difficulty in agreeing on the exact boundaries of trigger mentions (0.37), one of the major components of the frames. When the matching criterion is changed to allow mention overlap (approximate match), average question trigger agreement increases from 0.37 to 0.45, full frame agreement from 0.22 to 0.28, and core frame with type agreement from 0.41 to 0.44. While these figures are still low, they suggest that more strict guidelines may be useful in annotating triggers. In contrast to named entities which are often expressed with specific part-of-speech elements, triggers are lexically and syntactically more heterogeneous and ambiguous, especially in consumer health questions; thus, developing such guidelines may be difficult. Analyzing inter-annotator agreement for triggers of each question type, we find that the agreement is generally higher for triggers of the frequent question types (Table CAUSE (0.60), TREATMENT (0.52), PREVENTION (0.44), and PERSON_ORGANIZATION (0.43), while for the rare types, the agreement is very low or even zero (for SUPPORT_GROUP and OVERDOSE).Inter-annotator agreement in CHQA-email is overall low. The highest agreement is for question topics (0.71) and the lowest is for full frame agreement (0.22), which considers named entity and trigger mentions, and all semantic roles  . This isCOST, DIAGNOSE_ME, and COMPARISON, which are relatively rare, as well as TREATMENT, the third most frequent trigger type in this subset (Table TREATMENT).Agreement in CHQA-web is consistently higher than that in CHQA-email by about 0.2 points. Trends are similar for different semantic classes, confirming our basic intuition that annotating these shorter questions is easier, compared to CHQA-email. Agreement for named entities (0.72) is consistent with that reported for CHQ-NER (0.71) , althouget Table . AgreemeThe evaluation results of annotation confidence estimation for CHQA-web are provided in Table The results show that the baseline (AGREE: taking simple agreement of annotators as indication of confidence) already achieves an agreement of 0.79 with the adjudicated results. The precision of 0.99 indicates that a small number of annotations that the annotators agreed upon were discarded later in adjudication. All other methods improve on the low recall of the baseline (0.66), while leading to some precision loss. IAA_RANK uses the ranking of annotators by their average agreement score, and can be based on any of the agreement categories described earlier. Because frame annotations incorporate other atomic annotations , we ran IAA_RANK with two frame agreement results. The first ranks the annotators by their average agreement for full frame with triggers, and the second by their agreement for core frame with types. The latter led to best confidence estimation for CHQA-web (0.844).The results obtained with MACE  and P&C Full frame w/trigger and Core frame w/ type), MACE, and P&C subsets. In addition, we provide MACE and P&C confidence estimations for each annotation. This can enable further research in developing more advanced methods that estimate annotation confidence. For example, the confidence scores generated by MACE and P&C and the output of other simpler methods can be used as features to learn to predict confidence. Confidence scores can also be used in an active learning framework, where the annotator is presented only with the annotations with moderate confidence, reducing adjudication load. Distinguishing high from low confidence items can also benefit training and evaluation: for example, a system can be penalized less for an incorrect response, when the annotation is labeled with low confidence [We ran the same methods on CHQA-email (Unadjudicated) and provide this part of the corpus in several versions: AGREE, IAA_RANK  for a long time, since they were considered outside the scope of our work. More recently, we began considering these questions but annotated them only in CHQA-web. A large portion of the corpus has not been adjudicated due to personnel/time constraints, which may limit its usefulness. To address this, we experimented with several methods of annotation confidence estimation and make these estimates available, which could stimulate research in better understanding annotator behavior and automatic adjudication.Our corpus has some limitations. Our conceptualization of question understanding and our goals have evolved over time and the corpus reflects this to some extent. Two parts of the corpus are largely similar, but there are also differences. In most cases, these differences can be reconciled with some automatic processing, as in question types that are merged in the other part of the corpus or non-In ongoing and future work, we plan to expand this corpus on several other semantic dimensions. For example, we are currently normalizing named entities in our corpus by mapping them to the corresponding UMLS Metathesaurus concepts. By addressing the lexical variability in named entities, this can improve question understanding. In earlier work, we found that coreference played a major role in consumer health question understanding , and we