This paper considers the linear quadratic regulator (LQR) optimal control problem of multi-agent unmanned vehicle systems under communication constraints with packet drops. The problem is formulated into a distributed optimization problem of minimizing a global cost function through the sum of local cost functions by using local information exchange. By utilizing a newly developed optimization technique, we propose a novel algorithm to solve the distributed LQR problem in a first order (gradient descent based) manner. Moreover, we adopt the key idea of virtualizing an extra node for each agent to store information from the previous step and create a fully distributed optimization algorithm. Extensive simulations demonstrate the efficacy and robustness of the proposed solution.