In the field of remote sensing image compression it is often argued that traditional MSE-based fidelity metrics might not effectively describe the quality of remote sensing lossy or near-lossless compressed images. In this paper we introduce a performance evaluation framework based on both reconstruction fidelity and impact on image exploitation. Besides MSE, the framework also considers hard classification and mixed pixel classification, as well as anomaly detection. We apply this framework to evaluate and compare the quality of state-of-the-art lossy and near-lossless compression techniques applied to hyperspectral AVIRIS scenes.