In the development of modern gear transmission systems, the demand for high speed, heavy load, lightweight design, and low noise has become increasingly stringent. For herringbone gear systems, which are widely used in marine and aerospace applications due to their high load capacity and balanced axial thrust, understanding the influence of key design parameters such as contact ratio on nonlinear dynamic behavior is essential. In this study, we focus on the effect of contact ratio on the vibration characteristics of a herringbone gear system by establishing a comprehensive nonlinear dynamic model that incorporates time-varying mesh stiffness, mesh impact, and backlash. The herringbone gear is modeled as a 12-degree-of-freedom (DOF) bending-torsion-shaft coupled system. Through tooth contact analysis and loaded tooth contact analysis, we calculate the integrated and single tooth mesh stiffness while accounting for assembly errors. A novel corner meshing impact model that explicitly considers the contact ratio is proposed to compute the impact force during mesh engagement. By varying the tooth height modification coefficients, we change the contact ratio and analyze the resulting vibration responses. The results demonstrate that under constant load, increasing the contact ratio generally reduces the mesh circumferential and pinion axial vibrations; however, when the contact ratio exceeds a critical value (approximately 4.07 in our example), the vibration increases due to the enhanced nonlinear effect of backlash.
Figure 
1. Corner Meshing Impact Model Considering Contact Ratio
During the meshing process of gear teeth, the existence of transmission errors and tooth deflections leads to a deviation of the actual mesh point from the theoretical line of action. This deviation causes impact at the instant of engagement, known as corner meshing impact. In our study, we focus on the impact at engagement (mesh-in impact) because it has a more significant influence than mesh-out impact, as validated by Seireg and Houser. The impact arises due to the difference in tangential velocity between the mating teeth at the engagement point. To accurately compute the impact force, we propose a model that accounts for the fact that, in a herringbone gear with a high contact ratio, several tooth pairs are already in contact when a new pair enters mesh. The existing tooth pairs provide additional compliance that buffers the impact.
Figure 1 (conceptual diagram) illustrates the corner meshing impact model considering contact ratio. The kinetic energy at the impact point D is given by:
$$
E_k = \frac{1}{2} \frac{I_1 I_2 v_s^2}{I_1 r’^2_{b2} + I_2 r^2_{b1}} = \frac{1}{2} q_s \delta_s^2 + \frac{1}{2} q_r \delta_r^2
$$
where:
- \(E_k\) is the kinetic energy at impact
- \(I_1, I_2\) are the moments of inertia of the pinion and gear
- \(v_s\) is the impact velocity at point D
- \(r_{b1}\) is the base circle radius of the pinion under normal meshing
- \(r’_{b2}\) is the instantaneous base circle radius of the gear during corner meshing
- \(q_s\) is the compliance of the single tooth pair at the impact point, obtained from tooth contact analysis by dividing the integrated mesh stiffness by the load sharing factor
- \(q_r\) is the combined compliance of all other tooth pairs already in contact at the moment of impact
- \(\delta_s\) is the deformation of the impacting tooth pair
- \(\delta_r\) is the deformation of the remaining tooth pairs
- \(\theta\) is the angle between the instantaneous line of action at the impact point and the theoretical line of action. From geometry:
$$
\theta = \arccos\left( \frac{r’_{b2}}{r_{O2D}} \right) – \angle PO_2D – \alpha
$$
where \(r_{O2D}\) is the tip circle radius of the driven gear, and \(\alpha\) is the pressure angle.
The maximum impact force \(F_s\) can be derived as:
$$
F_s = \frac{v_s}{\sqrt{ \frac{I_1 I_2}{I_1 r’^2_{b2} + I_2 r^2_{b1}} (q_s + q_r \cos^2 \theta) }}
$$
To validate our model, we compare it with the model proposed by Zhou et al. (2008) which does not consider the buffering effect of other tooth pairs. The comparison is shown in Table 2 for five different contact ratios achieved by varying the pinion’s tooth height modification coefficient. The herringbone gear parameters used are listed in Table 1.
| Parameter | Pinion (Active) | Gear (Passive) |
|---|---|---|
| Normal module (mm) | 6 | 6 |
| Transverse pressure angle (°) | 20 | 20 |
| Helix angle (°) | 24.43 | 24.43 |
| Backlash (μm) | 2 | 2 |
| Load torque (N·m) | 800 | 800 |
| Damping ratio | 0.1 | 0.1 |
| Density (g/cm³) | 7.85 | 7.85 |
| Number of teeth | 17 | 44 |
| Hand of helix | Left / Right | Right / Left |
| Face width (mm) | 75 | 75 |
| Mass moment of inertia (kg·m²) | 0.065 | 3.70 |
| Operating speed (r/min) | 2000 | 772 |
| Pinion modification coefficient | Contact ratio ε | Impact force from Zhou et al. (kN) | Impact force from our model (kN) |
|---|---|---|---|
| +0.25 | 2.72 | 9.255 | 8.896 |
| +0.12 | 2.91 | 8.661 | 8.378 |
| 0 | 3.30 | 6.593 | 6.469 |
| -0.13 | 3.68 | 5.180 | 5.121 |
| -0.22 | 4.07 | 2.107 | 2.096 |
From Table 2, it is clear that our model predicts slightly lower impact forces (1% to 5% difference) compared to the reference model, especially for lower contact ratios. This is because when fewer tooth pairs are engaged, the combined compliance of the existing pairs is larger, providing a greater buffering effect. Our model explicitly accounts for this, leading to more accurate predictions. The impact force decreases with increasing contact ratio, which can be attributed to the increased overall mesh stiffness reducing the tooth deformation and thus the synthesis base pitch error causing the impact.
2. Dynamic Model of the Herringbone Gear System
For herringbone gear transmissions, the left and right helical gear pairs cannot achieve perfect symmetry in practice. Consequently, the system exhibits not only torsional and lateral vibrations but also axial motion, forming a bending-torsion-shaft coupled vibration system. In our model, we assume the pinion is floating. The system has 12 degrees of freedom, represented by the generalized displacement vector:
$$
\{ \delta \} = \{ y_{p1}, z_{p1}, \theta_{p1}, y_{g1}, z_{g1}, \theta_{g1}, y_{p2}, z_{p2}, \theta_{p2}, y_{g2}, z_{g2}, \theta_{g2} \}^T
$$
where \(y_{ij}, z_{ij}, \theta_{ij}\) (\(i=p,g; j=1,2\)) denote the translational and rotational displacements of the pinion and gear centers for the left (j=1) and right (j=2) helical halves. The system equations incorporate time-varying mesh stiffness, backlash, and mesh impact excitation. Using Newton’s laws, the equations of motion are:
For the left pinion:
$$
\begin{aligned}
m_{p1} \ddot{y}_{p1} + c_{p1y} \dot{y}_{p1} + k_{p1y} y_{p1} &= -F_{y1} \\
m_{p1} \ddot{z}_{p1} + c_{p12z} (\dot{z}_{p1} – \dot{z}_{p2}) + k_{p12z} (z_{p1} – z_{p2}) &= -F_{z1} \\
I_{p1} \ddot{\theta}_{p1} &= -F_{y1} \cdot R_p + T_{p1} – F_{s1} \cdot R_p
\end{aligned}
$$
For the left gear:
$$
\begin{aligned}
m_{g1} \ddot{y}_{g1} + c_{g1y} \dot{y}_{g1} + k_{g1y} y_{g1} &= F_{y1} \\
m_{g1} \ddot{z}_{g1} + c_{g1z} \dot{z}_{g1} + k_{g1z} z_{g1} + c_{g12z} (\dot{z}_{g1} – \dot{z}_{g2}) + k_{g12z} (z_{g1} – z_{g2}) &= F_{z1} \\
I_{g1} \ddot{\theta}_{g1} &= F_{y1} \cdot R_g – T_{g1} + F_{s1} \cdot R_g
\end{aligned}
$$
For the right pinion:
$$
\begin{aligned}
m_{p2} \ddot{y}_{p2} + c_{p2y} \dot{y}_{p2} + k_{p2y} y_{p2} &= -F_{y2} \\
m_{p2} \ddot{z}_{p2} – c_{p12z} (\dot{z}_{p1} – \dot{z}_{p2}) – k_{p12z} (z_{p1} – z_{p2}) &= -F_{z2} \\
I_{p2} \ddot{\theta}_{p2} &= -F_{y2} \cdot R_p + T_{p2} – F_{s2} \cdot R_p
\end{aligned}
$$
For the right gear:
$$
\begin{aligned}
m_{g2} \ddot{y}_{g2} + c_{g2y} \dot{y}_{g2} + k_{g2y} y_{g2} &= F_{y2} \\
m_{g2} \ddot{z}_{g2} + c_{g2z} \dot{z}_{g2} + k_{g2z} z_{g2} – c_{g12z} (\dot{z}_{g1} – \dot{z}_{g2}) – k_{g12z} (z_{g1} – z_{g2}) &= F_{z2} \\
I_{g2} \ddot{\theta}_{g2} &= F_{y2} \cdot R_g – T_{g2} + F_{s2} \cdot R_g
\end{aligned}
$$
In the above equations, \(m_p, m_g, I_p, I_g\) are the masses and moments of inertia of pinion and gear; \(R_p, R_g\) are the pitch circle radii; \(c_{p1y}, c_{g1y}, \dots\) and \(k_{p1y}, k_{g1y}, \dots\) are the equivalent support damping and stiffness of shafts and bearings; \(c_{p12z}, c_{g1z}, c_{g2z}, c_{g12z}, k_{p12z}, k_{g1z}, k_{g2z}, k_{g12z}\) are the axial damping and stiffness of gears and shafts. \(F_{y1}, F_{y2}, F_{z1}, F_{z2}\) are the tangential and axial dynamic mesh forces; \(F_{s1}, F_{s2}\) are the mesh impact forces. The expressions for the dynamic mesh forces are given by:
$$
\begin{aligned}
F_{y1} &= \cos\beta_1 \, c_{m1} \left[ \cos\beta_1 (\dot{y}_{p1} – \dot{y}_{g1} + R_p \dot{\theta}_{p1} – R_g \dot{\theta}_{g1}) + \sin\beta_1 (\dot{z}_{p1} – \dot{z}_{g1}) \right] \\
&\quad + \cos\beta_1 \, k_{m1} \, f \left[ \cos\beta_1 (y_{p1} – y_{g1} + R_p \theta_{p1} – R_g \theta_{g1}) + \sin\beta_1 (z_{p1} – z_{g1}) \right] \\
F_{z1} &= \sin\beta_1 \, c_{m1} \left[ \cos\beta_1 (\dot{y}_{p1} – \dot{y}_{g1} + R_p \dot{\theta}_{p1} – R_g \dot{\theta}_{g1}) + \sin\beta_1 (\dot{z}_{p1} – \dot{z}_{g1}) \right] \\
&\quad + \sin\beta_1 \, k_{m1} \, f \left[ \cos\beta_1 (y_{p1} – y_{g1} + R_p \theta_{p1} – R_g \theta_{g1}) + \sin\beta_1 (z_{p1} – z_{g1}) \right]
\end{aligned}
$$
Similar expressions hold for the right side (\(F_{y2}, F_{z2}\)) with \(\beta_2\) and \(k_{m2}, c_{m2}\). Here, \(\beta\) is the helix angle; \(k_{m1}(t), k_{m2}(t)\) are the time-varying normal mesh stiffness of the left and right gear pairs; \(c_{m1}, c_{m2}\) are the viscous mesh damping coefficients; \(f(\cdot)\) is a piecewise nonlinear function representing the backlash, defined as:
$$
f(x) =
\begin{cases}
x – b, & x > b \\
0, & |x| \le b \\
x + b, & x < -b
\end{cases}
$$
where \(b\) is half the backlash. Note that the mesh stiffness values we used incorporate installation errors and tooth profile modifications through tooth contact analysis, thereby integrating the error excitation into the stiffness excitation.
3. Analysis of Vibration Response under Different Contact Ratios
We non-dimensionalized the 12-DOF equations of motion and solved them using a variable-step fourth-order Runge-Kutta numerical integration method. The herringbone gear parameters from Table 1 were employed. The pinion’s modification coefficient was varied to obtain five different contact ratios: 2.72, 2.91, 3.30, 3.68, and 4.07. We focused on the mesh circumferential vibration (relative tangential displacement across the mesh) and the pinion axial vibration for the left helical half, as these are primary noise sources.
Figures 3–10 (replaced by descriptions) show the acceleration time histories and phase-plane plots for selected contact ratios. The root-mean-square (RMS) values of acceleration are summarized in Table 3.
| Contact ratio ε | Mesh circumferential acceleration RMS (m/s²) | Pinion axial acceleration RMS (m/s²) |
|---|---|---|
| 2.72 | 32.7 | 2.07 |
| 2.91 | — | — |
| 3.30 | 15.8 | 1.02 |
| 3.68 | — | — |
| 4.07 | 34.4 | 2.56 |
When the contact ratio increased from 2.72 to 3.30, the mesh circumferential RMS acceleration dropped from 32.7 m/s² to 15.8 m/s², and the pinion axial RMS acceleration decreased from 2.07 m/s² to 1.02 m/s². This reduction is due to the increased mesh stiffness and the consequent reduction in impact force. The system exhibited nearly linear periodic behavior because the load-induced tooth deflection was sufficient to overcome the nonlinearity of backlash. However, when the contact ratio reached 4.07, both accelerations increased markedly (34.4 m/s² and 2.56 m/s²). The phase-plane plots at ε = 4.07 (Figures 8 and 10) show a complex attractor, indicating strong nonlinearity. At this high contact ratio, the mesh stiffness is so high that the external load cannot deform the teeth enough to eliminate the backlash gap, causing repeated impacts and chaotic-like behavior. Thus, there exists an optimal contact ratio below which vibration decreases with increasing contact ratio, and above which vibration increases due to the prominence of backlash nonlinearity.
Our results demonstrate the importance of selecting the appropriate contact ratio for a herringbone gear system in practice. For the given load case (800 N·m), a contact ratio around 3.30 provides significant noise and vibration reduction compared to a lower value. However, blindly increasing the contact ratio beyond a critical threshold can worsen the dynamic performance.
4. Conclusions
In this work, we developed a comprehensive nonlinear dynamic model for a herringbone gear system and analyzed the influence of contact ratio on its vibration characteristics. The key findings are:
- We proposed a corner meshing impact model that accounts for the buffering effect of existing tooth pairs. Our model yields impact forces slightly lower (1%–5%) than previous models, especially for low contact ratios, and shows good agreement with reference data.
- Under a constant load of 800 N·m, increasing the contact ratio from 2.72 to 3.30 significantly reduced the mesh circumferential and pinion axial vibrations. The system exhibited periodic behavior with low acceleration levels.
- When the contact ratio was further increased to 4.07, the vibration levels increased substantially. This is attributed to the fact that the high mesh stiffness prevents the load from closing the backlash gap, resulting in strong nonlinear dynamics and impact-induced vibrations.
- The existence of an optimal contact ratio suggests that gear designers should carefully balance the benefits of high contact ratio (smooth transmission) with the detrimental effects of backlash nonlinearity under specific load conditions.
Our study provides a theoretical basis for selecting contact ratios in herringbone gear systems to achieve low noise and vibration. Future work will extend the model to include variations in load, speed, and tooth modifications for a more comprehensive design guide.