It’s web-based (server side PHP using MySql database). There’s no icon on your desktop to click. To make posts, etc. you log in to the admin panel on your WordPress site.
To add new plugins and themes, or upload file changes, most people download these to their desktop, unzip them, and then FTP them to the site, although there’s increasing ability to do these things from within the online admin panel in the latest versions. (Your mileage may vary.)
Re: altering the design of a WordPress installation. If you are customizing your design, rather than simply installing a new theme, how you go about it is a matter of personal preference. Many files can be altered within the admin panel, although I personally don’t find this useful at all. What I do (and I suspect most people do) is use an HTML/CSS editor to test out changes in the CSS files before uploading (I use static versions of the generated HTML files to do this).
Other people may also install a local copy of their site on their personal machine (you’ll need something like Xaamp to do this in order to process the PHP and set up a local MySql database) and work on changes to their site locally before uploading.
There are lots and lots of ways to customize an installation, but you will have to become familiar with WordPress, the use of WordPress templates, and some PHP knowledge is also very useful.
What is it? It’s an open-source blogging application. It has enough flexibility to function as a Content Management System (CMS) and there are many, many plugins out there to facilitate whatever you want to do with it.
Start looking at existing WordPress sites, and you’ll see the many different things people do with it.
I think covers that everything.