Вернулся в Лихай. После недели на NeurIPS, такое чувство, что меня здесь не было целый год. Будто бы вернулся из будущего в прошлое, где никто не говорит про LLMs, diffusion models, equivariant neural networks, autoencoders, ODEs, PDEs и т.д. 👀

Еще, на NeurIPS мне посчастливилось встретиться с нашими казахстанскими рисерчерами. Прикрепляю их работы снизу. Все - первые авторы🔥

Landscape Surrogate: Learning Decision Losses for Mathematical Optimization Under Partial Information
Arman Zharmagambetov · Brandon Amos · Aaron Ferber · Taoan Huang · Bistra Dilkina · Yuandong Tian

Exploring Foveation and Saccade for Improved Weakly-Supervised Localization
Timur Ibrayev · Manish Nagaraj · Amitangshu Mukherjee · Kaushik Roy

Algorithm Selection with Priority Order for Instances
Zhamilya Saparova · Martin Lukac

One of the best things I rediscovered this year was meditation.

I got involved in it through the Waking Up App – and I think it’s the best one on the market. Besides guided meditations, it teaches you a theory about different types of meditation philosophies like stoicism Buddhism, helps you go through challenging times, etc.

The subscription is paid, but you can get it for free just by asking for it :)

Also, you can get free 30 days through this link (I don’t benefit anything from it): https://dynamic.wakingup.com/shareOpenAccess/SC55A93B4

I am not in the position to say this, but AI research is turning into a numbers game of how many papers you can publish, regardless of their quality and novelty. I heard this idea a lot recently and from pretty reputable faculty members.

https://www.argmin.net/p/too-much-information

Sam Harris is a master of a good word ☝️. This one just came in as a daily reminder:

The whole point of life is not to arrive, but to begin again.

Because everything you’ve done, or not done, is now just a memory.

And everything you are telling yourself about the future, is a half-truth at best.

Because however good or bad things seem: nothing lasts.

Because there’s so much more to life than talking to yourself, about yourself.

Because this moment, is wide open. The world itself, is wide open.

And no one knows how good life can be.

Because in this moment, there’s always an opportunity, to stop looking over your own shoulder.

In this, living instance, and only here, this is where you can lose the mask that you’ve mistaken for your own face.

This next moment it’s truly new and totally unexplored,
And that’s always the case.

So, wherever you’ve come from, however hard things have seem, whoever is in your life, and no longer in it.

Whatever has been done, or left undone,
Just begin again…

Looking back to 2023, this was a year of mental growth rather than a professional or academic one. Sure enough, career-wise, I achieved several important things, like getting a full-time job, getting my paper published on NeurIPS, making a few vital industry connections, etc. – however, mostly, these achievements were a result of my long-term effort, starting from my freshman year.

This year, the quality of my thoughts and my inner talk got significantly better. As the year progressed, I realized that the most profound changes often happen quietly. While my professional milestones were publicly acknowledged and celebrated, the internal growth I experienced was a personal journey deeply rooted in introspection, self-awareness, and a few experiences I would not publicly discuss here. However, this growth was mainly nurtured by a few essential practices I integrated into my daily life.

Firstly, I adopted mindfulness meditation. This practice helped me cultivate a sense of presence and awareness. I learned to observe my thoughts without judgment, understanding that they don’t define me. This was particularly helpful in stressful situations, allowing me to respond with more clarity and composure.

Secondly, I started hitting the gym more frequently and consistently. Daily morning exercise sets me up for the day and encourages me to tackle challenging tasks throughout the day. The primary source of inspiration for me was Peter Attia’s book Outlive and his “The Drive” podcast.

Thirdly, I understood that regardless of the milestones I achieve, my mind will always try to find problems to overcome. That does not sound bad at first glance, but it’s the basis of the hedonic treadmill concept, which suggests that we quickly return to a relatively stable level of happiness despite major positive or negative events or life changes. I realized that success alone wouldn’t sustain my long-term happiness or fulfillment. Furthermore, problems our mind creates are likely not problems – they are just artificial boundaries. Now, whenever I get upset by some obstacle ahead of me, I ask myself, "Is this a problem? I bet millions of people worldwide would hope this is their problem." This brings me gratitude and a completely different view of the situation. Instead of focusing on what I don’t have or what could be better, I started appreciating what I already possess. This shift in mindset brought a profound sense of fulfillment and helped me stay grounded, especially during challenging times.

In conclusion, 2023 was a transformative year for me. While my professional achievements were significant, the internal growth I experienced was far more impactful. I learned that personal development is a continuous journey, and I am excited to see where this path takes me in the years to come. The true measure of this year’s success is not just in what I accomplished, but in how I evolved as a person, both mentally and emotionally.

Looking ahead to 2024, I want to remind myself and others of one thing: “Life rewards those who take risks.” Looking forward to more risk taking in the upcoming year.

Просто хочу поделиться офигенным продуктом: https://metaphor.systems

Solves many engineering problems and headaches if you are building something with web parsing

Сейчас подумал, что я жалею, что не сделал study abroad during my undergrad. Думаю, это классный опыт, даже не смотря на то, что учеба в Америке – это уже и есть study abroad. Теперь так и буду думать какого-же это учиться где-то в Европе или Азии.

What I am reading/learning right now

Из CS/ML:

1. Engineering Math: Differential Equations and Dynamical Systems. Крутой YouTube курс от Steve Brunton, профессора в UW. Объясняет dynamical systems очень доступно. Он кстати выступал на NeurIPS и так я узнал про его канал.

2. Grokking Concurrency. Мне не хочется загружать себя в последний семестр и брать Parallel Systems, поэтому недавно нашел крутую книгу на эту тему.

3. Vector Databases: from Embeddings to Applications. Краткий курс про RAGs - хайповая тема которую хочу изучить.

Books:

1. Being You - Anil Seth. Сейчас слушаю аудиокнигу. Чел рассказывает про свою теорию consciousness.

2. Radical Acceptance - Tara Brach. Слышал много рекоммендаций этой книги.

3. The Creative Act - Rick Rubin. Рик Рубин напоминает какого-то старого, мудрого философа, который всегда говорит правильные вещи.

Если интересно, могу писать такие короткие посты на регулярной основе 🤝

У меня одного такая проблема в телеграмме, что я подписан на много интересных каналов, но нет времени их почитать? В результате собирается много непрочитанных сообщений.

I don’t think LLMs are path to AGI.

Particularly, I am not convinced (yet?) by the idea that everything can be interpreted with language, i.e. that consciousness in humans arises from language. It’s definitely a medium of high quality information crucial for building AGI; however, there are some modalities of information that are lost.

If millions of people told you what it is like to “fall in love”, would you be able to understand it fully without experiencing it? I doubt. There is no language that can describe the full experience of our inner self.

A side hustle for you all:

https://www.rexby.com/travel-guides

Btw follow Inboxblend on Twitter (X)! We post some cool news about AI, startups, science every two hours 👇

https://x.com/inboxblend

I’m officially 2nd Dan black belt in Goju-Ryu Karate now 🥋🤓

This is so cool. Especially using Kahneman’s thinking fast, and slow idea is so neat. I even remember expressing a similar idea in one of my SOPs for PhD programs (that I never submitted).

AlphaGeometry is a neuro-symbolic system made up of a neural language model and a symbolic deduction engine, which work together to find proofs for complex geometry theorems. Akin to the idea of “thinking, fast and slow”, one system provides fast, “intuitive” ideas, and the other, more deliberate, rational decision-making.

Because language models excel at identifying general patterns and relationships in data, they can quickly predict potentially useful constructs, but often lack the ability to reason rigorously or explain their decisions. Symbolic deduction engines, on the other hand, are based on formal logic and use clear rules to arrive at conclusions. They are rational and explainable, but they can be “slow” and inflexible - especially when dealing with large, complex problems on their own.

https://deepmind.google/discover/blog/alphageometry-an-olympiad-level-ai-system-for-geometry/

Arc – is my favorite browser. Lightweight and fast, much better than Chrome. I was recently experiencing an issue on my Mac where a single open PDF in one of my tabs in Chrome, kept the entire browser on hold. Switched to Arc, and nothing similar ever happened again. Enjoy: https://arc.net/gift/6debefe1