Tumor vascularity is an important factor that has been shown to correlate with tumor malignancy and was demonstrated as a prognostic indicator for a wide range of cancers. Three-dimensional (3-D) power Doppler ultrasound (PDUS) offers a convenient tool for investigators to inspect the signals of blood flow and vascular structures in breast cancer. In this paper, a new computer-aided diagnosis (CAD) system for quantifying Doppler ultrasound images based on 3-D thinning algorithm and neural network is proposed. We extracted the skeleton of blood vessels from 3-D PDUS data to facilitate the capturing of morphological changes. Nine features including vessel-to-volume ratio, number of vascular trees, length of vessels, number of branching, mean of radius, number of cycles, and three tortuosity measures, were extracted from the thinning result. Benign and malignant tumors can therefore be differentiated by a score computed by a multilayered perceptron (MLP) neural network using these features as parameters. The proposed system was tested on 221 breast tumors, including 110 benign and 111 malignant lesions. The accuracy, sensitivity, specificity, and positive and negative predictive values were 88.69% (196/221), 91.89% (102/111), 85.45% (94/110), 86.44% (102/118), and 91.26% (94/103), respectively. The Az value of the ROC curve was 0.94. The results demonstrate a correlation between the morphology of blood vessels and tumor malignancy, indicating that the newly proposed method can retrieves a high accuracy in the classification of benign and malignant breast tumors.