Psalms 33:1 (HCSB)

 Rejoice in the Lord, you righteous ones;
praise from the upright is beautiful.

I love this translation!  Praise from the upright is beautiful because it fits with the purpose of the universe.  All things—ALL THINGS—were created to praise and glorify their creator.  This is good, right, and acceptable.  If one cannot glorify one’s maker, whom can one glorify?  God is worthy of all praise and honor, by all of God’s creation. Thus, praise is beautiful coming from God’s people.
We are upright, not because we are worthy, but because God became worthy for us in Christ.  
What else can this all do if not fill us with joy?
Spurgeon: That God is, and that he is such a God, and our God, ours for ever and ever, should wake within us an unceasing and overflowing joy.