Hey there, @shridsan2010! Thanks for contacting us. I’m happy to help you.
Creating the sidebar itself is something that can change depending on the theme you are using. Some themes have a sidebar by default, others don’t, but you can add it, but the steps really depend on the theme.
By default, on WooCommerce, it is possible to add a category list on the sidebar, by using the Product Categories List block.
However, it won’t have the list of products the way the link you shared has.
But you can achieve this with third-party plugins.
I found one that I believe can help you achieve your goal:
Product Dropdown Widget for WooCommerce.
If you want to look for other options, you can search on the Plugins > Add New section of your dashboard or on a search engine like Google, for example.
Please note that we do not provide support for third-party plugins, so it is good to check the plugin thoroughly to see if it is reliable. You can find more about this in this guide.
I hope this was clarifying. Please let us know if there’s anything else we can do to help or if you have any questions.
Have a wonderful day!